apple-perf #381

Run time

Learn about OS pricing on GitHub Actions

Job	Run time
set-parameters	4s
prepare-test-specs (llama, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114...	42s
prepare-test-specs (llama, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538511...	42s
prepare-test-specs (llama, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device...	42s
prepare-test-specs (mv3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	42s
prepare-test-specs (mv3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	40s
prepare-test-specs (mv3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	40s
prepare-test-specs (mv2, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	40s
prepare-test-specs (mv2, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	40s
prepare-test-specs (mv2, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	44s
prepare-test-specs (ic4, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	41s
prepare-test-specs (ic4, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	41s
prepare-test-specs (ic4, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	39s
prepare-test-specs (ic3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	43s
prepare-test-specs (ic3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	43s
prepare-test-specs (ic3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	40s
prepare-test-specs (resnet50, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385...	40s
prepare-test-specs (resnet50, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538...	44s
prepare-test-specs (resnet50, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:dev...	41s
prepare-test-specs (edsr, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	40s
prepare-test-specs (edsr, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114...	42s
prepare-test-specs (edsr, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep...	42s
prepare-test-specs (mobilebert, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:3085353...	40s
prepare-test-specs (mobilebert, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535...	40s
prepare-test-specs (mobilebert, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	39s
prepare-test-specs (w2l, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	41s
build-benchmark-app / macos-job	18m 44s
prepare-test-specs (w2l, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	40s
prepare-test-specs (w2l, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	42s
prepare-test-specs (meta-llama/Llama-3.2-1B, llama3_fb16, apple_iphone_15, arn:aws:devicefarm:us-...	42s
prepare-test-specs (meta-llama/Llama-3.2-1B, llama3_coreml_ane, apple_iphone_15+ios_18, arn:aws:d...	40s
prepare-test-specs (meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8, llama3_spinquant, apple_...	40s
prepare-test-specs (meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8, llama3_qlora, apple_iphone_1...	40s
export-models (edsr, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devi... / macos-job	6m 46s
export-models (edsr, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	7m 3s
export-models (edsr, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:0... / macos-job	5m 29s
export-models (ic3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	6m 36s
export-models (ic3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	6m 1s
export-models (ic3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	6m 36s
export-models (ic4, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	7m 5s
export-models (ic4, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	5m 49s
export-models (ic4, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	7m 29s
export-models (llama, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:dev... / macos-job	8m 52s
export-models (llama, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:... / macos-job	7m 50s
export-models (llama, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devi... / macos-job	8m 18s
export-models (meta-llama/Llama-3.2-1B, llama3_coreml_ane, apple_iphone_15+ios_18, arn:aws:device... / macos-job	16m 19s
export-models (meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8, llama3_qlora, apple_iphone_15, ar... / macos-job	9m 50s
export-models (meta-llama/Llama-3.2-1B, llama3_fb16, apple_iphone_15, arn:aws:devicefarm:us-west-... / macos-job	9m 41s
export-models (mobilebert, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538511... / macos-job	7m 11s
export-models (meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8, llama3_spinquant, apple_iphon... / macos-job	8m 42s
export-models (mobilebert, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	7m 1s
export-models (mobilebert, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / macos-job	9m 3s
export-models (mv2, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	6m 33s
export-models (mv2, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	5m 26s
export-models (mv2, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	6m 24s
export-models (mv3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	6m 35s
export-models (mv3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	5m 20s
export-models (mv3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	6m 26s
export-models (resnet50, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / macos-job	6m 30s
export-models (resnet50, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo... / macos-job	5m 28s
export-models (w2l, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	6m 28s
export-models (resnet50, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d... / macos-job	8m 6s
export-models (w2l, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	7m 45s
export-models (w2l, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	5m 31s
upload-benchmark-app	38s
benchmark-on-device (edsr, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / mobile-job (ios)	46m 11s
benchmark-on-device (edsr, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	31m 4s
benchmark-on-device (edsr, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538511... / mobile-job (ios)	31m 18s
benchmark-on-device (ic3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	22m 38s
benchmark-on-device (ic3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	30m 49s
benchmark-on-device (ic3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	22m 21s
benchmark-on-device (ic4, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	1h 24m 17s
benchmark-on-device (ic4, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	13m 2s
benchmark-on-device (ic4, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	1h 1m 16s
benchmark-on-device (llama, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:3085353851... / mobile-job (ios)	34m 0s
benchmark-on-device (meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8, llama3_qlora, apple_iphone_... / mobile-job (ios)	41m 58s
benchmark-on-device (llama, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538511... / mobile-job (ios)	21m 31s
benchmark-on-device (llama, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / mobile-job (ios)	1h 37m 24s
benchmark-on-device (meta-llama/Llama-3.2-1B, llama3_coreml_ane, apple_iphone_15+ios_18, arn:aws:... / mobile-job (ios)	13m 41s
benchmark-on-device (meta-llama/Llama-3.2-1B, llama3_fb16, apple_iphone_15, arn:aws:devicefarm:us... / mobile-job (ios)	20m 23s
benchmark-on-device (meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8, llama3_spinquant, apple... / mobile-job (ios)	37m 2s
benchmark-on-device (mobilebert, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853... / mobile-job (ios)	26m 48s
benchmark-on-device (mobilebert, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	44m 31s
benchmark-on-device (mobilebert, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535... / mobile-job (ios)	13m 2s
benchmark-on-device (mv2, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	30m 45s
benchmark-on-device (mv2, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	32m 27s
benchmark-on-device (mv2, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	12m 50s
benchmark-on-device (mv3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	32m 8s
benchmark-on-device (mv3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	12m 33s
benchmark-on-device (mv3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	22m 21s
benchmark-on-device (resnet50, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:3085353... / mobile-job (ios)	13m 12s
benchmark-on-device (resnet50, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:de... / mobile-job (ios)	30m 48s
benchmark-on-device (w2l, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	30m 45s
benchmark-on-device (w2l, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	28m 24s
benchmark-on-device (resnet50, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538... / mobile-job (ios)	29m 26s
benchmark-on-device (w2l, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	46m 43s
upload-benchmark-results	1m 49s
	21h 26m 18s

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

apple-perf #381

Usage

apple-perf #381

Jobs

Run details

Run time