Snowflake Lambda Data Loader – Example with AWS S3 Trigger
This is an example of how to make an AWS Lambda Snowflake database data loader. Snowflake database is a cloud platform suited to working with large amounts of data for data warehousing and analysis. AWS Lambda provides serverless compute – or really what is server…
How To Load Data Into Snowflake – Snowflake Data Load Best Practices
On one hand it is straightforward how to load data into Snowflake, on the other there are some Snowflake data loading best practices one should follow to leverage the architecture most efficiently. This article will primarily focus on ingest testing and scaling, however there are…
How To Extract Snowflake Data – Unload Best Practices
What’s the best way to extract data out of Snowflake? I’m unloading it from Snowflake to S3 and am curious of how to maximize performance. An AWS lambda function I’m working on will pick up the data for additional processing. Single File Extract The test…
Snowflake Database Now Available on Azure
No surprise, it was bound to happen – retailers competing against Amazon certainly aren’t keen on the idea of giving Amazon money for their computing services! Snowflake was recently announced as available on the Microsoft Azure platform and is no longer exclusive only to the…
Database and Query Tuning with Snowflake Clustering Keys
Snowflake Clustering Keys seem to have been a more recently introduced, but extremely powerful, feature of the Snowflake database. They’re a simple feature with a large impact on query tuning and run-time query performance, particularly during scans. There’s a clever concept in Netezza called zone…
Snowflake Database Architecture
Snowflake is a database built from scratch from the cloud – as a result, unlike others that were not, they were able to start without the burden of any traditional architecture and make the best no compromise decisions in designing the Snowflake architecture. This is…
Cluster Key Performance Impact on Snowflake Joins
I’ve read that Snowflake clustering helps improve join speeds, although I’m not sure how just being in order would make that the case exactly. I’m going to try some tables to get a decent demonstrable fact-dimension join; sf1000 lineitem (6 billion rows, 169 GB) and…
This Snowflake Database Looks Like The Real Deal
In 2006, I was invited into a meeting room at AC Nielsen where I was consulting at the time. Nielsen had just purchased their first Netezza system. I believe Mark Jeske was Netezza’s first sales engineer, a role I would later play myself at IBM….
IBM Cloud Image Classification, or Using Watson to Detect Rice
IBM’s Platform As A Service (PAAS) IBM Cloud makes it suuuuuper simple to make something like an image classifier. As I’m new to playing with any of this stuff, just the high level concept and usage is fun enough to start playing around with. It’s…
IBM Cloud / Bluemix / Cloud Lite Account
IBM was rolling out Bluemix when I was working there, which has now been renamed to IBM Cloud. Bluemix is probably just a subset of it today is my guess. As a Platform As A Service (PAAS) it provides a space for you to just…
Stay connected