Streaming Twitter data into Snowflake

Data Pipeline

data pipeline

Step 1

Step 2

pip3 install tweepy

Code:

Step 3

CREATE STAGE stage_name 
URL = 's3://bucket-name'
CREDENTIALS = (AWS_KEY_ID = 'xxx' AWS_SECRET_KEY = 'yyy');
CREATE TABLE table_name (v variant);
CREATE PIPE pipename 
COMMENT = 'pipe_description'
AUTO_INGEST = true
AS
COPY INTO table_name FROM @stage_name
FILE_FORMAT = (TYPE = 'JSON');

an odd brief blip