Migrate Your Knowledge and AI Workloads to Databricks With the AWS Migration Acceleration Program


    On this weblog we outline the method for incomes AWS buyer credit when migrating Knowledge and AI workloads to Databricks on Amazon Internet Companies (AWS) with the AWS Migration Acceleration Program (MAP). We are going to present you the best way to use AWS MAP tagging to determine new migrated workloads comparable to Hadoop and Enterprise Knowledge Warehouses (EDW), with a view to guarantee workloads qualify for helpful AWS buyer credit. This info is useful for patrons, technical professionals at know-how and consulting companions, in addition to AWS Migration Specialists and Answer Architects.

    Databricks overview

    Databricks is the information and AI firm. Greater than 7,000 organizations worldwide — together with Comcast, Condé Nast, H&M and over 40% of the Fortune 500 — depend on the Databricks Lakehouse Platform to unify their information, analytics and AI. Based by the unique creators of Apache Spark™, Delta Lake and MLflow, Databricks is on a mission to assist information groups remedy the world’s hardest issues. Databricks is acknowledged by Gartner as a Chief in each Cloud Database Administration Methods and Knowledge Science and Machine Studying Platforms.

    The Databricks Lakehouse on AWS unifies the very best of knowledge warehouses and information lakes in a single easy platform to deal with all of your information, analytics and AI use circumstances. It’s constructed on an open and dependable information basis that effectively handles all information varieties and applies one widespread safety and governance method throughout all your information and cloud platforms.

    What’s the AWS Migration Acceleration Program (MAP)?

    The AWS Migration Acceleration Program (MAP) is a complete and confirmed cloud migration program primarily based upon AWS’s expertise migrating 1000’s of enterprise prospects to the cloud. Enterprise migrations will be complicated and time-consuming, however MAP will help you speed up your cloud migration and modernization journey with an outcome-driven methodology.

    MAP offers instruments that scale back prices and automate and speed up execution via tailor-made coaching approaches and content material, experience from AWS Skilled Companies, a world associate community, and AWS funding. MAP additionally makes use of a confirmed three-phased framework (Assess, Mobilize, and Migrate and Modernize) that can assist you obtain your migration targets. By way of MAP, you’ll be able to construct robust AWS cloud foundations, speed up and scale back threat, and offset the preliminary value of migrations. Leverage the efficiency, safety, and reliability of the cloud.

    Why do you’ll want to tag assets?

    Migrated assets should be recognized with a selected map-migrated tag (tag secret’s case delicate) to make sure AWS credit are supplied to prospects as an incentive and to scale back the price of migrations. The tagging course of defined under needs to be used for Hadoop, Knowledge Warehouse, on-premises, or different cloud workload migrations to AWS.

    Steps to Tag Migrated Sources

    The next infographic offers an outline of the seven-step course of:

    7-step process for implementing AWS MAP tagging in Databricks on AWS

    Arrange an AWS Group account

    Setting up an AWS Organization account for use with Databricks on AWS

    Arrange a Databricks Workspace

    Arrange your Databricks workspacethrough Cloud Formation or the Databricks account console in lower than quarter-hour.

    Set up your Databricks workspace via Cloud Formation or the Databricks account console in less than 15 minutes.

    Activate AWS MAP Tagging

    Present the Migration Program Engagement ID (MPE ID is obtained after signing an AWS MAP Settlement together with your AWS representatives) on the CloudFormation stack for use to create the dependent AWS objects. This may create Value and Utilization Reviews (CUR) and generate a server ID for use by the AWS Migration Hub for migrations.

    AWS CloudFormation template for producing server IDs and establishing Value and utilization studies

    AWS CloudFormation template for generating server IDs and setting up Cost and usage reports

    Offering the MPE ID earlier than initiating the AWS CloudFormation Stack for MAP

    Providing the MPE ID before initiating the AWS CloudFormation Stack for MAP

    After the AWS CloudFormation is run efficiently, copy the migration hub server IDs generated from the output and tag them as a worth to the map-migrated tag set on the Databricks clusters used because the goal clusters for migration. Along with Databricks clusters, observe the identical tagging mechanism throughout different AWS assets used for the migration, together with the Amazon S3 buckets and Amazon Elastic Block Retailer (EBS) volumes.

    Copying the server IDs from the AWS CloudFormation output for use in MAP tagging

    Copying the server IDs from the AWS CloudFormation output to be used in MAP tagging

    Databricks clusters getting used for migration

    Databricks clusters being used for migration

    Spin up the Databricks clusters for migration and tag them with map-migrated tags considered one of 3 ways: 1. the Databricks console, 2. the AWS console, or 3. the Databricks’ API and its cluster insurance policies.

    1. MAP tagging Databricks clusters utilizing the Databricks console (most popular)

    MAP tagging Databricks clusters using the Databricks console (preferred)

    Amazon EBS volumes are mechanically MAP tagged when tagging is completed through the Databricks console/h4>

    Amazon EBS volumes are automatically MAP tagged when tagging is done via the Databricks console

    2. MAP tagging Databricks clusters through the AWS console

    MAP tagging Databricks clusters via the AWS console

    3. Databricks cluster tagging will be carried out through cluster insurance policies

    Remember to tag the related Amazon S3 buckets

    Databricks cluster tagging can be performed via cluster policies

    As soon as all Databricks on AWS assets are tagged appropriately, carry out the migration and monitor the utilization through AWS Value Explorer. Organizations who’ve signed an AWS MAP Settlement and carried out all of the required steps will see credit utilized to their AWS account. Keep in mind to activate the MAP tags within the Value Allocation Tags part of the AWS Billing Console. The map-migrated tags might take as much as 24 hours to indicate up within the Value Allocation Tags part after you’ve got deployed the CloudFormation template.

    Once all Databricks on AWS resources are tagged appropriately, perform the migration and track the usage via AWS Cost Explorer.

    Activating Value Allocation Tags

    Activating Cost Allocation Tags

    Robotically Delivered Value and Utilization Reviews

    Companies > Billing > Value & Utilization Reviews.

    Automatically Delivered Cost and Usage Reports


    On this weblog we defined the best way to efficiently tag migrated workloads to Databricks on AWS utilizing the AWS Migration Acceleration Program (MAP). Utilizing tags to determine migrated workloads will profit prospects via AWS credit. The steps concerned embrace producing server IDs on the AWS Migration Hub, establishing value allocation tags, utilizing MAP tags to focus on Databricks clusters, mechanically delivering value and utilization studies, and monitoring utilization through Value Explorer.

    Questions? E mail us at [email protected].

    Extra Sources

    AWS Migration Acceleration Program (MAP)

    Hadoop Migrations

    SAS Migrations

    Knowledge Warehouse Migrations


    Please enter your comment!
    Please enter your name here