The new concepts introduced here are warehouses and credits.Įssentially a warehouse is how you specify the power of compute that you use to run queries. These include roles, users, databases and warehouses. Once you’ve signed up, you’ll need a few things in place as part of the deployment. DeploymentĪs of publishing this, you can sign up and get started with a free (no credit card), month-long trial, which gets you floating. We’ll then apply some structure to loading, after getting the security and costs watertight we will finally set sail with some interesting new features and capabilities. We will start with an intro to a Snowflake deployment. The structure of this post will loosely follow the order in which you’ll encounter and want to consider various new concepts and features as you implement Snowflake. The context also caters entirely towards doing your transforming tasks in a SQL transformation tool like Dataform. Things to be kept in mind when doing the initial deployment. This post doesn’t get too far into the details of the doing, but rather points out things that are somewhat peculiar or unique to Snowflake. However, the most platformy move here is a direct integration with Salesforce. This is made clear in their recent manoeuvring into the crowded, polluted sea of Data Marketplaces, and a peek into the BI world, with their very simple new Dashboards tool. Last part of the intro fanfare: Snowflake is a Data Platform. S3 buckets + EC2 for anyone feeling like they’d rather DIY this part, or build a competitor. At some point, it should start to become clear that Snowflake is just a clever interface for storage and computation built on commodity cloud infrastructure. Snowflake offers a choice of AWS, Azure or GCP for your horsepower, so that might be reason enough for you to choose Snowflake. I’m no expert, but you’d struggle to go wrong with either. The primary alternative to Snowflake in this context is Google Bigquery. But you’ll still be writing SQL queries, in a mostly familiar pleasant SQL syntax. It stores data in a columnar way (rather than rows), which means it is very fast. A very fast one, that handles loads of data, and has lots of usability features. Pretty standard SQL otherwise, and a few new concepts.Īnd you only pay for the capacity and performance you use. If you’ve used PostgreSQL then this shouldn’t feel too foreign, minus index maintenance, table locks, performance issues and upgrades. Putting useful things in your path and keeping anything and everything operationally complex out of your way. Snowflake’s real value is the reduction of non-value-adding complexity for the user. Nonetheless, we may need some justification. However, this post doesn't get too far into why Snowflake. Though with the flowery praise to come it should be (see contacts below). Welcome to my pocket notebook, heading Snowflake Important Things - Jan 2020. These few notes, scripts and points of reference should save you some time and get you out onto the water sooner. Specifically, the things I’ve noted as useful when implementing Snowflake. In this post, we’ll be taking a look into my notebook on storing. In my first post, I justified an approach to achieve a scalable system for loading, storing, transforming and distributing data within an analytics context.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |