diff options
| author | Santo Cariotti <santo@dcariotti.me> | 2025-01-13 19:08:36 +0100 |
|---|---|---|
| committer | Santo Cariotti <santo@dcariotti.me> | 2025-01-13 19:09:09 +0100 |
| commit | 80930bb9d945b2ffee0fdda78ebd8cbe1caa4dc2 (patch) | |
| tree | 589ff2610bab1c2d2facabc103974179d0c289e9 /README.md | |
| parent | e6d14f7b388f4f866234d444668d8801cbf9661c (diff) | |
Partitions number as argument
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 10 |
1 files changed, 5 insertions, 5 deletions
@@ -64,7 +64,7 @@ To test on Google Cloud, execute the following shell scripts in the given order: - `scripts/02-dataproc-copy-jar.sh` - `scripts/03-update-network-for-dataproc.sh` - `scripts/04-dataproc-create-cluster.sh <num-workers> <master-machine> <worker-machine>` -- `scripts/05-dataproc-submit.sh` +- `scripts/05-dataproc-submit.sh <num-partitions>` - `scripts/06-dataproc-update-cluster.sh <num-workers> <master-machine> <worker-machine>` - `scripts/07-cleanup.sh` @@ -93,13 +93,13 @@ $ scripts/00-create-service-account.sh; \ scripts/02-dataproc-copy-jar.sh; \ scripts/03-update-network-for-dataproc.sh; \ scripts/04-dataproc-create-cluster.sh 1 n1-standard-4 n1-standard-4; \ - scripts/05-dataproc-submit.sh; \ + scripts/05-dataproc-submit.sh 200; \ scripts/06-dataproc-update-cluster.sh 2 n1-standard-4 n1-standard-4; \ - scripts/05-dataproc-submit.sh; \ + scripts/05-dataproc-submit.sh 200; \ scripts/06-dataproc-update-cluster.sh 3 n1-standard-4 n1-standard-4; \ - scripts/05-dataproc-submit.sh; \ + scripts/05-dataproc-submit.sh 200; \ scripts/06-dataproc-update-cluster.sh 4 n1-standard-4 n1-standard-4; \ - scripts/05-dataproc-submit.sh + scripts/05-dataproc-submit.sh 200 ``` After that, you can also check the created 4 jobs. |