Ask or search…
K
Links

Snowflake

Loading data into Snowflake via external stages.
This guide will walk you through the process of exporting data from your DCN into Snowflake using Google Cloud Storage (GCS), Amazon S3, or Azure Blob Storage as an intermediary storage bucket. This leverages Snowflake's external stage feature, which allows you to load data stored in an external stage into a Snowflake table, which can then be re-loaded into your DCN via the Snowflake source.

Prerequisites

Before proceeding, ensure that you have the following:
Please note that this guide assumes you have the necessary permissions to create and manage resources in Snowflake and your chosen cloud storage service. Always ensure data is handled in a secure and compliant manner.

Step 1 - Export Data from your DCN

The first step is to export the data from your DCN into the previously created Google Cloud Storage (GCS), Amazon S3, or Azure Blob Storage bucket in Parquet format. Once your audience is exported, you can then proceed to step 2.
Exporting data from your DCN in Parquet format.

Step 2 - Configure your external stage & load your data.

Amazon S3:

Example command:
COPY INTO my_table FROM @my_s3_stage FILE_FORMAT = (TYPE = 'PARQUET');

Google Cloud Storage (GCS):

Example command:
COPY INTO my_table FROM @my_gcs_stage FILE_FORMAT = (TYPE = 'PARQUET');

Azure Blob Storage:

Example command:
COPY INTO my_table FROM @my_azure_stage FILE_FORMAT = (TYPE = 'PARQUET');

Step 3 - Query Data in Snowflake

With the data successfully copied into Snowflake's internal tables (as detailed in Step 2), you are now ready to execute queries on these tables just like any other Snowflake table. To enhance your experience, consider utilizing Snowflake's features such as indexing, clustering, or other performance optimization techniques, especially when dealing with large datasets.
Following successful data export and querying, you can now perform further data analysis, create data visualizations, or develop machine learning models using Snowflake's comprehensive set of tools.
This guide provided a brief overview of the process. For more detailed steps and additional information, please refer to the official Snowflake documentation.
Last modified 6mo ago