Docker image+tag scan failing #647

setchy · 2023-10-18T15:14:42Z

When running cdxgen against public docker image+tags, I am repeatably receiving either an empty BOM, or a HTTPError: Response code 404 (Not Found)

Example commands

cdxgen -t docker docker.io/library/node:18.0.0 -o bom.json

cdxgen -t docker docker.io/library/eclipse-temurin:11.0.0-alpine -o bom.json

The text was updated successfully, but these errors were encountered:

prabhu · 2023-10-20T11:44:57Z

node bin/cdxgen.js -t docker docker.io/library/node:18.0.0 -o /tmp/bom.json                                                                                                                                                         ok 
Docker service in rootless mode detected.
Trying to pull the image docker.io/library/node:18.0.0 from registry. This might take a while ...
Trying with library/node
About to export image docker.io/library/node:18.0.0 to /tmp/docker-images-xQdjHA
Image docker.io/library/node:18.0.0 successfully exported to directory /tmp/docker-images-xQdjHA
Extracting layer 4de9349b11d6ab1a32ec1ad2683b9469f1804710f70e588542a91080b0efda19/layer.tar to /tmp/docker-images-xQdjHA/all-layers
Extracting layer 4240dd01392d300f41df7f7db411a7177151089fb21c94cfedf66227e96b935e/layer.tar to /tmp/docker-images-xQdjHA/all-layers
Extracting layer 6f058e66a7dcef48c51344ef0a3370858d42c3a1d2beccd2a580c573d8049305/layer.tar to /tmp/docker-images-xQdjHA/all-layers
Extracting layer c8d5427acf26c9968e414ff950dd7730f51e4d218e3953d9a3403dcb7ba6e488/layer.tar to /tmp/docker-images-xQdjHA/all-layers
Extracting layer 6e11d63ea5e15c1cd3ea1dde8005bd09c412eaabac3986b6468ef037f89472f1/layer.tar to /tmp/docker-images-xQdjHA/all-layers
Extracting layer 9594d541bb9fc83e998a1f7e4397608f5295dc51d19241d4a40500c6834b7da4/layer.tar to /tmp/docker-images-xQdjHA/all-layers
Extracting layer 0b4dc39f23e524fef34f09860e0c116100605eb9988c3e7c05cbec77aa2c83c1/layer.tar to /tmp/docker-images-xQdjHA/all-layers
Extracting layer 5432c8deeb49d340236628eba89d42c5c9b79608df6adffd2c20cab494a8c7bd/layer.tar to /tmp/docker-images-xQdjHA/all-layers
Extracting layer 9b0591382fa40c635049a07559e08efdc62acd1023d245a5779b536df827da5c/layer.tar to /tmp/docker-images-xQdjHA/all-layers
pathList [
  '/tmp/docker-images-xQdjHA/all-layers/usr/local/go',
  '/tmp/docker-images-xQdjHA/all-layers/usr/local/lib',
  '/tmp/docker-images-xQdjHA/all-layers/usr/local/lib64',
  '/tmp/docker-images-xQdjHA/all-layers/opt',
  '/tmp/docker-images-xQdjHA/all-layers/home',
  '/tmp/docker-images-xQdjHA/all-layers/usr/share',
  '/tmp/docker-images-xQdjHA/all-layers/usr/src',
  '/tmp/docker-images-xQdjHA/all-layers/var/www/html',
  '/tmp/docker-images-xQdjHA/all-layers/var/lib',
  '/tmp/docker-images-xQdjHA/all-layers/mnt',
  '/tmp/docker-images-xQdjHA/all-layers/usr/lib',
  '/tmp/docker-images-xQdjHA/all-layers/usr/lib64'
]

The first one worked for me, although I suspect it might be due to an existing image in the cache. The second one does fail.

prabhu · 2023-10-20T12:05:42Z

@setchy Any idea which registry has eclipse-temurin:11.0.0-alpine ? Unable to find it here https://hub.docker.com/_/eclipse-temurin/tags?page=1&name=11.0.0-alpine

11-alpine works.

node bin/cdxgen.js -t docker docker.io/library/eclipse-temurin:11-alpine -o /tmp/bom.json                                                                                                                                    ok  4s 
Docker service in rootless mode detected.
Trying to pull the image docker.io/library/eclipse-temurin:11-alpine from registry. This might take a while ...
Trying with eclipse-temurin:11-alpine
About to export image docker.io/library/eclipse-temurin:11-alpine to /tmp/docker-images-PoYhrB
Image docker.io/library/eclipse-temurin:11-alpine successfully exported to directory /tmp/docker-images-PoYhrB
Extracting layer afd2ec152c1bb9e40973ee0bfade03c146c1a9be79a02ff430caf3b1713fbaca/layer.tar to /tmp/docker-images-PoYhrB/all-layers
Extracting layer ae89a5eb8afb969a614a7c3b5db8d0b44f83a25956e431ae835a3c06bd42bc33/layer.tar to /tmp/docker-images-PoYhrB/all-layers
Extracting layer b151220b588f08629428fa3debf26a4289ade49c880eb02c4a92dae0d415308f/layer.tar to /tmp/docker-images-PoYhrB/all-layers
Extracting layer ff7b60bb4ba35391f0b3b4d27c2fe45f02702fa3f6551b116a5cc90ac01e340d/layer.tar to /tmp/docker-images-PoYhrB/all-layers
Extracting layer 8d7dc6857fc3fbba070dafb4a59f92606e377771156e01be797f798a64e4bf95/layer.tar to /tmp/docker-images-PoYhrB/all-layers
pathList [
  '/tmp/docker-images-PoYhrB/all-layers/usr/local/go',
  '/tmp/docker-images-PoYhrB/all-layers/usr/local/lib',
  '/tmp/docker-images-PoYhrB/all-layers/usr/local/lib64',
  '/tmp/docker-images-PoYhrB/all-layers/opt',
  '/tmp/docker-images-PoYhrB/all-layers/home',
  '/tmp/docker-images-PoYhrB/all-layers/usr/share',
  '/tmp/docker-images-PoYhrB/all-layers/usr/src',
  '/tmp/docker-images-PoYhrB/all-layers/var/www/html',
  '/tmp/docker-images-PoYhrB/all-layers/var/lib',
  '/tmp/docker-images-PoYhrB/all-layers/mnt',
  '/tmp/docker-images-PoYhrB/all-layers/usr/lib',
  '/tmp/docker-images-PoYhrB/all-layers/usr/lib64'
]

setchy · 2023-10-20T12:15:19Z

Ah interesting, looks like that publisher removes old/previous tags.

This is the error I get when doing the same...

~ % cdxgen -t docker docker.io/library/eclipse-temurin:11-alpine -o bom.json
HTTPError: Response code 404 (Not Found)
    at Request._onResponseBase (file:///Users/asetch/.nvm/versions/node/v20.8.1/lib/node_modules/@cyclonedx/cdxgen/node_modules/got/dist/source/core/index.js:706:31)
    at Request._onResponse (file:///Users/asetch/.nvm/versions/node/v20.8.1/lib/node_modules/@cyclonedx/cdxgen/node_modules/got/dist/source/core/index.js:768:24)
    at ClientRequest.<anonymous> (file:///Users/asetch/.nvm/versions/node/v20.8.1/lib/node_modules/@cyclonedx/cdxgen/node_modules/got/dist/source/core/index.js:786:23)
    at Object.onceWrapper (node:events:629:26)
    at ClientRequest.emit (node:events:526:35)
    at ClientRequest.emit (node:domain:488:12)
    at HTTPParser.parserOnIncomingClient (node:_http_client:693:27)
    at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)
    at Socket.socketOnData (node:_http_client:535:22)
    at Socket.emit (node:events:514:28) {
  input: undefined,
  code: 'ERR_NON_2XX_3XX_RESPONSE',
  timings: {
    start: 1697803934902,
    socket: 1697803934902,
    lookup: 1697803934902,
    connect: 1697803934902,
    secureConnect: undefined,
    upload: 1697803934902,
    response: 1697803934904,
    end: 1697803934905,
    error: undefined,
    abort: undefined,
    phases: {
      wait: 0,
      dns: 0,
      tcp: 0,
      tls: undefined,
      request: 0,
      firstByte: 2,
      download: 1,
      total: 3
    }
  },
  options: {
    request: undefined,
    agent: { http: undefined, https: undefined, http2: undefined },
    h2session: undefined,
    decompress: true,
    timeout: {
      connect: undefined,
      lookup: undefined,
      read: undefined,
      request: undefined,
      response: undefined,
      secureConnect: undefined,
      send: undefined,
      socket: undefined
    },
    prefixUrl: '',
    body: undefined,
    form: undefined,
    json: undefined,
    cookieJar: undefined,
    ignoreInvalidCookies: false,
    searchParams: undefined,
    dnsLookup: undefined,
    dnsCache: undefined,
    context: {},
    hooks: {
      init: [],
      beforeRequest: [],
      beforeError: [],
      beforeRedirect: [],
      beforeRetry: [],
      afterResponse: []
    },
    followRedirect: true,
    maxRedirects: 10,
    cache: undefined,
    throwHttpErrors: true,
    username: '',
    password: '',
    http2: false,
    allowGetBody: false,
    headers: {
      'user-agent': 'got (https://github.com/sindresorhus/got)',
      'accept-encoding': 'gzip, deflate, br'
    },
    methodRewriting: false,
    dnsLookupIpVersion: undefined,
    parseJson: [Function: parse],
    stringifyJson: [Function: stringify],
    retry: {
      limit: 2,
      methods: [ 'GET', 'PUT', 'HEAD', 'DELETE', 'OPTIONS', 'TRACE' ],
      statusCodes: [
        408, 413, 429, 500,
        502, 503, 504, 521,
        522, 524
      ],
      errorCodes: [
        'ETIMEDOUT',
        'ECONNRESET',
        'EADDRINUSE',
        'ECONNREFUSED',
        'EPIPE',
        'ENOTFOUND',
        'ENETUNREACH',
        'EAI_AGAIN'
      ],
      maxRetryAfter: undefined,
      calculateDelay: [Function: calculateDelay],
      backoffLimit: Infinity,
      noise: 100
    },
    localAddress: undefined,
    method: 'GET',
    createConnection: undefined,
    cacheOptions: {
      shared: undefined,
      cacheHeuristic: undefined,
      immutableMinTimeToLive: undefined,
      ignoreCargoCult: undefined
    },
    https: {
      alpnProtocols: undefined,
      rejectUnauthorized: undefined,
      checkServerIdentity: undefined,
      certificateAuthority: undefined,
      key: undefined,
      certificate: undefined,
      passphrase: undefined,
      pfx: undefined,
      ciphers: undefined,
      honorCipherOrder: undefined,
      minVersion: undefined,
      maxVersion: undefined,
      signatureAlgorithms: undefined,
      tlsSessionLifetime: undefined,
      dhparam: undefined,
      ecdhCurve: undefined,
      certificateRevocationLists: undefined
    },
    encoding: undefined,
    resolveBodyOnly: false,
    isStream: true,
    responseType: 'text',
    url: URL {
      href: 'http://unix/var/run/docker.sock:/images/docker.io/library/eclipse-temurin:11-alpine/get',
      origin: 'http://unix',
      protocol: 'http:',
      username: '',
      password: '',
      host: 'unix',
      hostname: 'unix',
      port: '',
      pathname: '/var/run/docker.sock:/images/docker.io/library/eclipse-temurin:11-alpine/get',
      search: '',
      searchParams: URLSearchParams {},
      hash: ''
    },
    pagination: {
      transform: [Function: transform],
      paginate: [Function: paginate],
      filter: [Function: filter],
      shouldContinue: [Function: shouldContinue],
      countLimit: Infinity,
      backoff: 0,
      requestLimit: 10000,
      stackAllItems: false
    },
    setHost: true,
    maxHeaderSize: undefined,
    signal: undefined,
    enableUnixSockets: true
  }
}
Manifest file /var/folders/yk/hjzy106n79b6y7d1bc9rnsfw0000gp/T/docker-images-V2swM2/manifest.json was not found after export at /var/folders/yk/hjzy106n79b6y7d1bc9rnsfw0000gp/T/docker-images-V2swM2
BOM generation has failed due to problems with exporting the image

Perhaps this is due to the archetype of my machine performing the pull/manifest retrieval (M2 arm chipset)

~ % docker pull eclipse-temurin:11-alpine           
11-alpine: Pulling from library/eclipse-temurin
no matching manifest for linux/arm64/v8 in the manifest list entries

setchy · 2023-10-20T12:23:54Z

When repeating with eclipse-temurin:21_35-jre-alpine

cdxgen fails with the same HTTP 404 error and manifest not found error
manually running docker pull eclipse-temurin:21_35-jre-alpine succeeds
Re-running cdxgen generates the bom.

Looks like it works if it's already within the local docker image cache...

prabhu · 2023-10-20T12:27:10Z

When repeating with eclipse-temurin:21_35-jre-alpine
* cdxgen fails with the same HTTP 404 error and manifest not found error

* manually running `docker pull eclipse-temurin:21_35-jre-alpine` succeeds

* Re-running cdxgen generates the bom.
Looks like it works if it's already within the local docker image cache...

Yes making a small improvement to this. The reason was the missing tag.

prabhu · 2023-10-20T13:01:34Z

@setchy could you test with the PR branch #656?

setchy · 2023-10-20T13:18:07Z

Still failing for me... Will work IF i manually pull the image prior to running cdxgen...

setchy · 2023-10-20T13:24:29Z

Odd, it's kinda working now but not convinced it isn't still cached somewhere.

Let me test with a wider set of images and report back...

setchy · 2023-10-20T13:38:42Z

Still getting HTTPError: Response code 404 (Not Found) if I haven't done a manual docker pull.

Tested using

node bin/cdxgen.js -t docker python:3.8.18-slim -o bom-python.json

prabhu · 2023-10-20T14:52:21Z

what do you mean by 404 error? Can you share the full output from this branch?

setchy · 2023-10-20T15:10:08Z

Same as I shared above. though its now working for a selection of python tags. will continue to monitor

prabhu · 2023-10-20T15:42:56Z

Same as I shared above. though its now working for a selection of python tags. will continue to monitor

Please take the latest and try again. This must be a recent change in behavior in docker.

prabhu self-assigned this Oct 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker image+tag scan failing #647

Docker image+tag scan failing #647

setchy commented Oct 18, 2023

prabhu commented Oct 20, 2023

prabhu commented Oct 20, 2023 •

edited

Loading

setchy commented Oct 20, 2023

setchy commented Oct 20, 2023

prabhu commented Oct 20, 2023

prabhu commented Oct 20, 2023

setchy commented Oct 20, 2023

setchy commented Oct 20, 2023 •

edited

Loading

setchy commented Oct 20, 2023

prabhu commented Oct 20, 2023

setchy commented Oct 20, 2023

prabhu commented Oct 20, 2023

Docker image+tag scan failing #647

Docker image+tag scan failing #647

Comments

setchy commented Oct 18, 2023

prabhu commented Oct 20, 2023

prabhu commented Oct 20, 2023 • edited Loading

setchy commented Oct 20, 2023

setchy commented Oct 20, 2023

prabhu commented Oct 20, 2023

prabhu commented Oct 20, 2023

setchy commented Oct 20, 2023

setchy commented Oct 20, 2023 • edited Loading

setchy commented Oct 20, 2023

prabhu commented Oct 20, 2023

setchy commented Oct 20, 2023

prabhu commented Oct 20, 2023

prabhu commented Oct 20, 2023 •

edited

Loading

setchy commented Oct 20, 2023 •

edited

Loading