BigQueryException: 404 Not Found

My BigQuery workspace can be seen below, showing my project, dataset and table:

I connected the integration with a service key that is linked to the GCP project. I also set the following permission for the service account to inherit down to the table level:

  • Owner

  • BigQuery Admin

  • BigQuery Job User

  • BigQuery User

My project can be found here: GitHub - codebrain001/layer_fraud_detection

Upon running my project, I have the following error:

Layer 0.8.9 using https://beta.layer.co
📁 Loading the project under /Users/codebrain/Documents/layer_fraud_detection
🔎 Found 1 dataset, 1 featureset and 1 model
📔 Session logs at /Users/codebrain/.layer/logs/20211023T105827-session-31b94416-8c
d1-4ff9-9e8f-849dbd980f99.log
💾 Starting at 2021-10-23 10:58:30...
🔵 Pipeline run id: 8c8aa627-5035-47f1-a335-93651a7e6a24
✅ 2021-10-23 10:58:30 | dataset     fraud_detection_data           ━━ DONE      …
⠦  2021-10-23 10:58:35 | featureset  fraud_detection_features       ━━ ERROR     …
                                     (DatasetBuildId(ab5b8c09-2d88-4e7b-8997-a5aa…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(20874e61-7fd1-4851-bdd0-814c…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(39ea691f-1646-4f52-8fc6-1771…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(584f3e7d-4504-4eea-95d1-8f7d…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(0948ced0-e0d0-40fb-8c09-3f0d…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(ccdc710d-9871-401e-8e8b-c5d9…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(afcdd1fc-8f2b-44d4-9ba3-0d3c…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(a1ec34b9-2348-4d48-a3d7-c109…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(11c44ec2-1369-4778-a6ce-1722…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(8c361361-301d-4dcb-b162-8e12…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(52f0eda5-ffa1-4aa4-ae07-1314…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(9d32442e-dbfd-4216-8583-b372…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(e067eb7f-0949-4aa5-b6f2-83c9…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(f045540f-c783-42a6-b041-ed00…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(802aa99d-122a-4e80-a07d-081b…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(bacce596-d201-4a40-be3c-2b9d…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     ),
                                     (DatasetBuildId(d5c94b4f-8cf7-433b-af85-9a0f…
                                     404 Not Found
                                     POST
                                     https://www.googleapis.com/upload/bigquery/v…
                                     {
                                       "error": {
                                         "code": 404,
                                         "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                         "errors": [
                                           {
                                             "message": "Not found: Dataset
                                     layer-ethereum-fraud-detection:transactions_…
                                             "domain": "global",
                                             "reason": "notFound"
                                           }
                                         ],
                                         "status": "NOT_FOUND"
                                       }
                                     }
                                     )
                                     Aborting...
⠦  2021-10-23 10:58:48 | model       fraud_detection_model          ━━ PENDING   …
LAYER RUN FAILED after 18161ms:
Failed to build featureset 'fraud_detection_features': (DatasetBuildId(ab5b8c09-2d8
8-4e7b-8997-a5aab5a8bfc1),com.google.cloud.bigquery.BigQueryException: 404 Not
Found
POST https://www.googleapis.com/upload/bigquery/v2/projects/layer-ethereum-fraud-de
tection/jobs?uploadType=resumable
{
  "error": {
    "code": 404,
    "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
    "errors": [
      {
        "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}
), (DatasetBuildId(20874e61-7fd1-4851-bdd0-814ceaa6309b),com.google.cloud.bigquery.
BigQueryException: 404 Not Found
POST https://www.googleapis.com/upload/bigquery/v2/projects/layer-ethereum-fraud-de
tection/jobs?uploadType=resumable
{
  "error": {
    "code": 404,
    "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
    "errors": [
      {
        "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}
), (DatasetBuildId(39ea691f-1646-4f52-8fc6-17716ba7a551),com.google.cloud.bigquery.
BigQueryException: 404 Not Found
POST https://www.googleapis.com/upload/bigquery/v2/projects/layer-ethereum-fraud-de
tection/jobs?uploadType=resumable
{
  "error": {
    "code": 404,
    "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
    "errors": [
      {
        "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}
), (DatasetBuildId(584f3e7d-4504-4eea-95d1-8f7d3768028c),com.google.cloud.bigquery.
BigQueryException: 404 Not Found
POST https://www.googleapis.com/upload/bigquery/v2/projects/layer-ethereum-fraud-de
tection/jobs?uploadType=resumable
{
  "error": {
    "code": 404,
    "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
    "errors": [
      {
        "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}
), (DatasetBuildId(0948ced0-e0d0-40fb-8c09-3f0dedf833e0),com.google.cloud.bigquery.
BigQueryException: 404 Not Found
POST https://www.googleapis.com/upload/bigquery/v2/projects/layer-ethereum-fraud-de
tection/jobs?uploadType=resumable
{
  "error": {
    "code": 404,
    "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
    "errors": [
      {
        "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}
), (DatasetBuildId(ccdc710d-9871-401e-8e8b-c5d94629c48d),com.google.cloud.bigquery.
BigQueryException: 404 Not Found
POST https://www.googleapis.com/upload/bigquery/v2/projects/layer-ethereum-fraud-de
tection/jobs?uploadType=resumable
{
  "error": {
    "code": 404,
    "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
    "errors": [
      {
        "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}
), (DatasetBuildId(afcdd1fc-8f2b-44d4-9ba3-0d3c018204e0),com.google.cloud.bigquery.
BigQueryException: 404 Not Found
POST https://www.googleapis.com/upload/bigquery/v2/projects/layer-ethereum-fraud-de
tection/jobs?uploadType=resumable
{
  "error": {
    "code": 404,
    "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
    "errors": [
      {
        "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}
), (DatasetBuildId(a1ec34b9-2348-4d48-a3d7-c10913c6fd07),com.google.cloud.bigquery.
BigQueryException: 404 Not Found
POST https://www.googleapis.com/upload/bigquery/v2/projects/layer-ethereum-fraud-de
tection/jobs?uploadType=resumable
{
  "error": {
    "code": 404,
    "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
    "errors": [
      {
        "message": "Not found: Dataset
layer-ethereum-fraud-detection:transactions_dataset",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}
)

I am keen on hearing from you, thanks.

Hi Brian,
What’s your dataset location? We currently support only EU, but working on improving that.

Okay, I used the US, making the changes now. I will update you on the results. Thanks, Yuranos.

I made the respective changes and it ran for approximately 11 hours and timed out.

layer start
Layer 0.8.9 using https://beta.layer.co
📁 Loading the project under /Users/codebrain/Documents/layer_fraud_detection
🔎 Found 1 dataset, 1 featureset and 1 model
📔 Session logs at /Users/codebrain/.layer/logs/20211025T095518-session-8b3a93b4-ab
cd-4cd8-ae21-0c4d89721546.log
💾 Starting at 2021-10-25 09:55:26...
🔵 Pipeline run id: d9e653ae-9a90-45df-a990-6772cdc45ac0
✅ 2021-10-25 09:55:26 | dataset     fraud_detection_data           ━━ DONE      …
⠋  2021-10-25 09:55:30 | featureset  fraud_detection_features       ━━ BUILDI
✅ 2021-10-25 09:55:26 | dataset     fraud_detection_data            DONE
⠙  2021-10-25 09:55:30 | featureset  fraud_detection_features        BUILD
✅ 2021-10-25 09:55:26 | dataset     fraud_detection_data            DONE
⠸  2021-10-25 09:55:30 | featureset  fraud_detection_features        BUIL
✅ 2021-10-25 09:55:26 | dataset     fraud_detection_data           ━━ DONE      …
✅ 2021-10-25 09:55:26 | dataset     fraud_detection_data           ━━ DONE      …
⠹  2021-10-25 09:55:30 | featureset  fraud_detection_features       ━━ BUILDING  …
⠹  2021-10-25 20:54:11 | model       fraud_detection_model          ━━ PENDING   …
code: UNAVAILABLE, details: Operation timed out

How can this be improved as I haven’t gotten a successful response.

I am still keen on getting updates, thanks🤓

Hey @CodeBrain ,

I saw that you created another issue for Model Catalog that the project uses the tables you are reporting here.
I wonder if Operation timed out issue is resolved or still ongoing?

Hi Brain,

Since we haven’t heard from you in a while, I will assume this issue is now resolved. Please don’t hesitate to open another thread if you are still facing issues.

Thanks,
Dimitar