About vedantroy-genmo

vedantroy-genmo

When running `df -h`, on a GCP batch VM, I see the following:```Filesystem Size Used Avail Use% Mounted on 2024-05-09 12:18:05.691 PDToverlay 28G 3.1G 25G 12% /``` Does this mean, I have around 25G of scratch space on a GCP batch VM, or ... is the co...

vedantroy-genmo

I created a VM in the same region as the my Google Cloud storage bucket, but when downloading from the bucket using rclone, the download speed is only ~ 263MiB/s. How can I get faster download speeds?I am using Tier 1 networking--but that doesn't see...

vedantroy-genmo

I restarted a VM on GCP, and now my local SSDs both appear mounted but empty. Is there any reason this would happen?

vedantroy-genmo

When using GCP batch, I'd like to download an "index" file at the start of a job and cache it on the local container's file system.But, this only works if the same container is used to run different tasks. Otherwise, if the container is restarted, an...

vedantroy-genmo

When using Google Cloud batch, I can mount my Google cloud storage bucket to a VM without issue. The programming running in the container successfully reads and writes from the bucket.I launched a VM with the goal of manually using gcsfuse to mount a...

vedantroy-genmo

Got it. And is the container itself restarted between tasks? In other words, if I download stuff to a temp dir inside the container, do I need to worry that the temp dir will be filled up overtime. Or, between each task, will the container get a fres...

vedantroy-genmo · 04-02-2024

Sure. I'm running the job in us-central1 (not sure if zone is us-central-1a, or multiple zones), and the UID is "internvid-md5-v1-a9339995-3599-48f6-b0".One issue is, I think the 125 is because of a spot pre-emption notice, but from reading the docs,...

vedantroy-genmo · 04-01-2024

Thanks, this worked. Is there anyway to select a specific field from the logs (to avoid large egress fees when analyzing?)Using `--format=json(textPayload)` works, but not sure if that is just doing the filtering on the client side (and thus still in...

vedantroy-genmo · 03-29-2024

Ok. If I search "Task task/" and "exited with status", I can see tasks that failed. The problem is, the error code is opaque.Task task/mytask-group0-281/0/0 runnable 0 exited with status 125 Do you have any thoughts on what might be happening?

vedantroy-genmo · 03-29-2024

Ok, this command sort of worked: gcloud beta batch tasks list --job my-job --project my-project --location us-central1 | grep FAIL But1. It outputs 2 rows, but I have 4 failed tasks2. It outputs the warning "ERROR: (gcloud.beta.batch.tasks.list) INVA...

My Stats

vedantroy-genmo's Bio

Badges vedantroy-genmo Earned

Recent Activity

Is all disk space shown with `df -h` available to gcp batch?

How to maximize download from Google Cloud Storage to VM?

Local SSDs are cleared when I restart a VM?

Will Google Cloud Batch persist the container file system between jobs?

How can I give my VMs write access to a cloud storage bucket (when using gcsfuse)?

Re: Will Google Cloud Batch persist the container file system between jobs?

Re: How can I figure out why a task failed on Google batch?

Re: How can I get logs from the command line for a batch job?

Re: How can I figure out why a task failed on Google batch?

Re: How can I view the ids of failed tasks?