Six Cloud Data Challenges Solved for App Developers

Cloud Data Challenges tackles common data issues you're likely facing as an app developer. Whether it's security, globalization, big data, AI, or something else, you're under pressure from management and your customers to deliver breakthrough experiences, and this e-book provides actionable answers for the hurdles in your path.

We wrote this book for developers focused on the requirements of their next cloud project and its critical data challenges. We've included the technical details specific to each scenario, and, if you're working these issues, you'll be familiar with the concepts we unpack and, hopefully, find value in the solutions we explore.

Are you getting the most out of the cloud?

If you're like most developers, you spend about two-thirds of your time building and the rest of your time learning. Sometimes your research is proactive, based on personal interest or a desire to maintain your skills, but more likely it's out of necessity—learning what you need to tackle your next big project.

Not long ago, new projects assumed an on-premises or hosted approach, forcing you to spend considerable time on essential requirements like scalability, availability, and security. Today, you still must deliver on those essentials, but the apps you're building need to do a lot more: support millions of users, span the globe, make sense of petabytes of data, and delight users in new ways.

Cloud services not only deliver "built-in" scalability and availability, but also provide an unprecedented range of prebuilt functionality for driving innovation into your apps—ready to evaluate for your next project, without your having to worry about physical infrastructure and the related overhead. The biggest challenge: there's so much out there that it can be hard to know where to start.

As a developer, you're probably already engaged with the cloud at some level, and you may be asking yourself, "Am I getting the most out of the data and AI technologies that are available in the cloud, or am I missing the key ingredients to deliver my next breakthrough application?"

If you're not sure of the answer to this question, you're not alone—especially when it comes to data. Microsoft internal research shows that, while 75 percent of developers are building data into their apps and 64 percent have a high level of confidence working with databases, only 24 percent of developers are comfortable activating cloud data services on their own.

This paper examines six of the most common data scenarios that you're likely to face and maps them to the services you might want to consider as a starting point:

  • Keeping everything up and running—how the cloud helps deliver on the essentials, such as scalability and availability, so that you can focus on what you do best: building apps.
  • Creating a multitenant software-as-a-service (SaaS) app that can effectively deliver the same services to hundreds or thousands of customers—while ensuring that they can't see one another's data.
  • Achieving global reach with your app—such as delivering fast, real-time recommendations to users across multiple countries and regions—without the hassles of complex configurations that span multiple datacenters.
  • Turning mountains of data into actionable insights through a combination of big data and advanced analytics.
  • Employing AI within your apps to reach out to users more naturally and contextually, wherever they are.
  • Ensuring security and compliance, regardless of what you're building in the cloud.

Like most developers, you have a myriad of tools and technologies at your disposal and want to use what works best for you, without compromising how you work. With Microsoft Azure, it's easy to integrate data and AI into your applications using your preferred data engine and the languages, frameworks, and tools of your choice. Read on to discover how to unlock the full potential of your data—wherever it lives, in whatever form it resides.

Scenario 1: Enterprise-proven services

No matter what your app does, you need to ensure that it delivers on the "essentials." Let's face it, every user demands a flawless experience with every interaction, including fast performance, high availability, ironclad security, and uncompromised privacy. But as a developer, while you know you need to deliver these core capabilities, you probably don't want to think about them—let alone be burdened with all the details necessary to ensure their delivery.

To determine whether you can trust a cloud platform to deliver on the basics—in a way that lets you stay flexible and focus on delivering new experiences that delight users—it's worth asking yourself a few questions:

  • How can the platform help me avoid negative user experiences like downtime? What does it offer that helps me avoid calls in the middle of the night or having to take my app offline every weekend for maintenance?
  • How can the platform help me handle unexpected spikes in workload and avoid complaints about poor performance—without having to pay for a lot of spare capacity that I might rarely need?
  • Can the platform help me expand to other geographic regions? And when it's time to do so, will I still have access to the same built-in availability, scalability, and security mechanisms that I've come to depend on?
  • How can the platform help me keep customer data secure? If there are regulatory or compliance requirements, does the platform have the necessary certifications?

More than any other factor, customer experiences determine whether companies thrive and profit, or struggle and fade. "Outside In: The Power of Putting Customers at the Center of Your Business" by Harley Manning and Kerry Bodine (New Harvest, 2012).

With your short-term investment in choosing the right cloud platform for your business, you'll soon be reaping the long-term rewards of stability, efficiency, and, ultimately, happy customers.

90 percent of Fortune 500 companies trust the Microsoft Cloud

Microsoft Azure services are built cloud-first—ready to deliver all the above, no matter where you are in the world. Azure runs on a worldwide network of Microsoft-managed datacenters, across 36 (and growing) regions, ensuring global coverage.

With Azure, you don't need to worry about capacity or performance. You can easily and instantly scale up when more capacity is needed, and then scale down to save money when demand subsides. You can configure Azure to let you know when capacity or performance need attention, or you can configure your Azure apps to scale up and down on their own.

Azure managed services help ensure availability through automatic patching and backups, built-in monitoring and security, and more, so you can stay focused on building great apps. Many Azure services are backed by service-level agreements (SLAs) that include uptime guarantees and downtime credit policies. To ensure business continuity and disaster recovery, you can choose from locally redundant storage (where data is replicated locally within your primary region) or geographically redundant storage, where data is replicated to a secondary region that's at least 250 miles (400km) away but within the same geography.

What's more, because Azure is available in 140 countries/regions, you can put your data where its users are. Storage of data can be restricted to a single geography, region, or country, so you always have control over how close your data and apps are to the people who use them.

Azure also provides comprehensive security and identity management tools and services, including advanced threat detection, Azure Security Center, Azure Active Directory, Azure Key Vault, and Azure Multi-Factor Authentication. And Azure offers the most comprehensive set of compliance offerings (including certifications and attestations) of any cloud service provider. (Security is such an important topic that we cover it separately later, in Scenario 6: Security and compliance.)

Whatever you're building, Azure helps ensure that your customers get enterprise-proven levels of scalability, availability, and security—everything your customers expect. And while Azure takes care of these fundamentals, you get the space to focus on the work you do best—development.

Figure 1. Microsoft Azure runs on a worldwide network of Microsoft-managed datacenters, across 36 (and growing) regions.

Customer case study: GeekWire

Based in Seattle, Washington, GeekWire is a rapidly growing technology news site with a global readership. Started by veteran tech journalists in 2011, GeekWire quickly established a reputation for fast-breaking industry news and expert analysis.

In the past, GeekWire ran on open-source, WordPress software with a MySQL database, and the site was hosted by a managed services provider dedicated to WordPress. Although that relationship worked well in the beginning, as GeekWire's popularity and site traffic increased, so did its performance concerns. "We rely heavily on people visiting us from social media and search engines, where performance is a ranking factor, so we need to deliver content very quickly," says Kevin Lisota, Web Developer at GeekWire.

Eventually, the time came for a better solution. Lisota and his team needed:

  • Increased control, including visibility into the underlying infrastructure and the ability to diagnose and fix issues themselves.
  • Instant scalability, so they could easily keep up with rapid growth, without any downtime or conversations with salespeople about "next tier" pricing.

When Lisota learned that Microsoft was launching a fully managed service for MySQL databases, moving to Azure made sense. "Azure Database for MySQL was what we were looking for," he says. "I don't have to deal with managing any of the details of the database—I don't have to patch, scale, or back it up and can rely on Microsoft to take care of that part of our infrastructure."

GeekWire's move to Azure delivered several immediate benefits, including on-demand scalability and reduced administrative overhead. "The biggest benefit of Azure Database for MySQL will be to have Microsoft manage and back up that resource for us so that we can focus on other aspects of the site," Lisota says. "Plus, we will be able to scale up temporarily as traffic surges and then bring it back down when it is not needed. That's a big deal for us."

Our page load times are very low, and we're able to do it on a more powerful and scalable infrastructure that costs us 45 percent less. Kevin Lisota, Web Developer, GeekWire

Read the entire GeekWire case study.

Hear from other customers who are using Azure.

For more information

Scenario 2: Multitenant SaaS applications

Software-as-a-service (SaaS) apps are becoming more and more popular across organizations of all sizes. This is happening for many reasons, including faster time-to-value, reduced up-front investment, and freedom from worrying about running and supporting apps.

To effectively build a SaaS app that serves hundreds or thousands of customers, delivers essential elements such as data isolation and security, and helps protect your margins, you may want to start by asking these key questions:

  • How can I provision new customer environments quickly—and keep performance high as overall usage grows? How will I handle varied and unpredictable spikes in demand?
  • How can I manage customer environments at scale, so that I won't need to onboard new staff linearly as my business grows?
  • How can I keep customer data secure—and isolate it to ensure that customers can't access one another's information? And how can I help my customers ensure secure access across their own user bases?
  • As the data in my app grows, how can I uncover relationships in the data and gain new insights from it—in areas ranging from app performance to customer behavior?
  • How can I help my customers visualize their data to gain new insights more quickly and easily— and thus add more value to my app?

If these questions resonate, then you probably already have a firm grasp of what you need from a cloud database solution.

Azure SQL Database: Redefining multitenancy

Azure SQL Database, an intelligent relational cloud database service, helps you avoid the typical tradeoffs between development efficiency, manageability, performance, and security for SaaS applications. As your app runs, SQL Database employs built-in machine learning to continuously assess your app's behavior, tune performance, and automatically improve reliability and data protection— freeing you to focus on other things. And as demand for your SaaS app grows, SQL Database scales on the fly, with virtually no app downtime.

With SQL Database, you can take advantage of:

  • Elastic database pools, which enable cost-effective and predictable pooling of resources to help you handle varied and unpredictable spikes in demand.
  • Elastic database jobs, which let you streamline admin tasks by making changes to many databases at once—ultimately enabling you to manage thousands of databases as one.
  • In-memory online transaction processing (OLTP), which keeps active data in main memory to improve performance by a factor of up to 30—giving you more performance for your money.
  • ColumnStore, which stores table data as columns instead of rows to deliver 10 times the data compression and queries that are up to 10 times faster.
  • Numerous ways to help keep customer data secure—including row-level security, dynamic data masking, and the Always Encrypted feature. Built-in intelligence to further boost app performance and data security—including features such as Threat Detection, Query Performance Insight, and SQL Database Advisor.
  • Graph processing, which can make it easier to express certain kinds of queries—and potentially improve query performance—by enabling you to model many-to-many relationships among relational data.

Azure SQL Database supports multiple data models for multitenant SaaS apps, so you can choose the one that delivers the optimal balance of isolation, cloud resource costs (compute and storage), and DevOps complexity for your unique situation. This also allows for a more flexible business model that includes a trial/free tier experience, serves a breadth of moderately active customers, and offers highend, dedicated premium offerings for your most demanding customers.

Figure 2. Azure SQL Database supports multiple data models for multitenant SaaS apps.

A list of all major Azure SQL Database features (with links to more information) can be found on the SQL Database features page.

Azure Active Directory

With a multitenant app, you need a way to organize and manage user accounts. More likely than not, your customers already have an identity management and authentication mechanism—and don't want to use and maintain a different one just to access your app.

You can use Azure Active Directory (Azure AD)—a multitenant, cloud-based directory and identity management service—to easily add single sign-on (SSO) to your SaaS app for greater value and differentiation in the marketplace. With Azure AD, you make it quick and simple for your customers to integrate with an existing Windows Server Active Directory, enabling them to leverage existing onpremises investments to manage access to your app.

Power BI Embedded

Few SaaS apps are complete without some sort of reporting environment. But all too often, such reports are static and deliver only limited insights. Embedding a rich, interactive analytics environment into your app is another way to increase its usefulness and help your customers make quick, datadriven decisions.

Power BI Embedded is an Azure service you can use to bring data to life within your SaaS app—without the time and expense of building and maintaining your own analytics environment. You can easily embed stunning, fully interactive reports in a cost-effective and scalable way—providing yet another source of value and differentiation, and potentially an additional revenue stream if you choose to position such functionality as a premium offering. You don't need to change the design of your app to use Power BI Embedded, and your users can sign in to your application just like before.

Customer case study: Umbraco

Umbraco is a popular open-source content management system. To deliver new customer value, Umbraco used Azure SQL Database and other Microsoft Azure services to build Umbraco-as-a-Service (UaaS), a multitenant SaaS app that lets customers self-provision new environments.

To make new customer environments available virtually immediately, Umbraco maintains a pool of pre-provisioned databases using elastic database pools. As the pre-provisioned databases are used, new elastic database pools are created and new databases are pre-provisioned. An automated process restores databases that are no longer needed to the elastic database pool. As of October 2016, Umbraco had nearly 3,000 databases across 19 elastic database pools.

Remarkably, Umbraco doesn't employ any database administrators. "We wanted to spend our time on solving our customers' problems, not managing infrastructure," says Niels Hartvig, founder of Umbraco. "We initially considered hosting the servers ourselves, but capacity planning would have been a nightmare."

By choosing Azure, Umbraco is able to provide customers with optimal performance, without the investment in IT resources that a self-hosted solution would require. "We love the developer convenience and scalability that Azure gives us, and our customers are thrilled with the features and reliability," says Morten Christensen, Technical Lead at Umbraco.

Elastic database pools are a perfect fit for our SaaS offering because we can dial capacity up and down as needed. Provisioning is easy, and with our setup, we can keep utilization at a maximum. Morten Christensen, Technical Lead, Umbraco

Read the entire Umbraco case study.

Hear from other customers who are using Azure SQL Database.

For more information

Scenario 3: Personalization around the globe

In today's global economy, apps and their users span the planet. To support rich, real-time personalization at global scale, your app needs fast access to the data that drives this personalization regardless of where the user is located.

To achieve this, from a data perspective, you need to consider how to:

  • Manage and version complex schemas.
  • Scale both throughput and storage based on global demand.
  • Balance the tradeoffs between the limited consistency models in a distributed database— including latency, availability, throughput, and scalability. (Most databases only offer two models, consistent and eventual, forcing you to choose from one end of the spectrum or the other.)
  • Deliver customized, real-time, and highly responsive experiences.
  • Ensure an always-on system around the globe.

In addition, to deliver the greatest value, you need to handle a variety of unstructured data at scale. Social media posts, mobile data, document text, application telemetry, and website content are just a few examples of today's data that doesn't always fit neatly into rows and columns. You also need to collect and analyze the data needed to drive real-time personalization before you can operationalize it for real-time access.

Ideally, you need a way to meet all these requirements without the hassle of deploying and managing servers, complex configurations of multiple datacenters across several regions, and complex big data, advanced analytics, and machine learning technology to drive personalization.

Azure Cosmos DB: A globally distributed, multi-model database service

Azure Cosmos DB is designed to provide low latency access to rich data anywhere in the world— making real-time personalization at global scale exactly the type of scenario that Azure Cosmos DB is designed to support. What's more, with Azure Cosmos DB, you can model your data the way your app requires, using familiar APIs, tools, and frameworks.

Azure Cosmos DB provides:

  • Turnkey global distribution—Azure Cosmos DB is currently the only fully managed database service to offer turnkey global distribution, letting you put your data where your users are for fast, responsive access. Every database account can be associated with any number of Azure regions, and the data replicates automatically, synchronously, and durably.
  • Support for multiple modes and APIs—Only Azure Cosmos DB empowers you with key-value, graph, and document data in one service. Azure Cosmos DB automatically indexes all data, so you don't need to worry about schema or index management. You also have your choice of APIs— including SQL, JavaScript, Gremlin, MongoDB, and Azure Table storage.
  • Limitless elastic scale around the globe—With Azure Cosmos DB, you can independently and elastically scale storage and throughput at any time, anywhere across the globe, paying only for the throughput and storage you need.
  • Multiple, well-defined consistency choices—Azure Cosmos DB offers an intuitive programming model and is currently the only non-relational database service to offer five well-defined consistency levels—enabling you to build for the unique needs of your app and best navigate the tradeoffs between consistency, latency, availability, throughput, and scalability.
  • Guaranteed low latency—With its latch-free and write-optimized database engine, Azure Cosmos DB guarantees less than 10-ms latencies on reads and less than 15-ms latencies on (indexed) writes at the ninety-ninth percentile.
  • Industry-leading, enterprise-grade SLAs—Azure Cosmos DB is currently the only cloud database service to offer industry-leading SLAs for 99.99 percent availability, latency at the ninetyninth percentile, and guaranteed throughput and consistency.

The capabilities provided by Azure Cosmos DB make it well suited for many use cases beyond realtime personalization at global scale—including IoT and telematics, retail and marketing, gaming, web and mobile applications, banking, and other mission-critical, global scenarios.

Azure Functions: Enabling a serverless architecture

By taking advantage of the native integration between Azure Cosmos DB and Azure Functions, you can quickly and easily build and deploy event-driven, serverless, personalized apps that have lowlatency access to rich data on a global scale. Because Azure Functions are event-driven, you can simply listen to a change feed from Azure Cosmos DB instead of creating your own listening logic. You can also bind an Azure Function to an Azure Cosmos DB collection using an input binding (which reads data from a container when a function executes) or using an output binding (which writes data to a container when a function completes).

Figure 3. Native integration between Azure Cosmos DB and Azure Functions makes it easy to build fast, serverless, personalized apps at global scale.

With Azure Functions, there are no limits—you set the parameters and the functions execute in parallel, with the Azure Functions service spinning up as many as times you need. The service creates new instances of functions whenever an event fires and closes them as soon as the function completes. This makes functions good for quick tasks and enables you to only pay for the time your functions are running.

Used together, Azure Cosmos DB and Azure Functions deliver a compelling set of benefits: eventdriven, serverless computing at near-infinite global scale, with low-latency access to rich data for serverless apps, app performance that enables a real-time user experience, and freedom from infrastructure through fully managed services.

Obtaining and analyzing the data to drive real-time personalization

You need to collect and analyze the data needed to drive real-time personalization before you can operationalize it and push it into Azure Cosmos DB for real-time access. The Azure big data and advanced analytics services you can use to do this—such as Azure HDInsight for analyzing data in Spark or Hadoop environments and Microsoft Power BI to bring that analysis to life—are covered later, in Scenario 4: Big data and advanced analytics.

Customer case study: Next Games

Next Games, maker of the popular mobile strategy game The Walking Dead: No Man's Land, has been with Azure from the start. By 2017, No Man's Land had 16 million installations, had generated 120 GB of new data each day, and was handling 11,500 database requests per second. To support continued growth, Next Games needed more storage, increased flexibility, and the ability to maintain availability and performance for users who aren't geographically close to one another.

To meet those needs, the company adopted Azure Cosmos DB. "We want our game data to always reside in the datacenter closest to the player so that we can load the player data as fast as possible," says Kalle Hiitola, Chief Technology Officer at Next Games. "[The Azure Cosmos DB] replication feature allows us to do this, so that we always have the data close by the player, no matter which datacenter it comes from."

Currently, all player data is in Azure Cosmos DB except for the actual saved player file, which is stored in Azure Blob storage. In addition to Azure Cosmos DB and, soon, Service Fabric, Next Games is using Media Services for streaming video, Event Hubs for analytics, Notification Hubs to send push notifications to clients, Traffic Manager for load balancing, and Web Apps for the dashboard, which provides game management.

The good thing about Azure is that we can incrementally add new features Microsoft brings on to Azure into our platform and switch them behind the scenes. You just start a new service, and off you go exploring what it can do for you. Kalle Hiitola, Chief Technology Officer, Next Games

Read the entire Next Games case study.

Hear from other customers who are using Azure Cosmos DB.

For more information

Scenario 4: Big data and advanced analytics

Customers today have high expectations when it comes to speed, consistency, personalization, and more. While failing to meet those expectations might cause customers to leave, simply delivering what they already expect might not be enough to delight them. So how can you exceed their expectations?

The answer lies in your data—or, more accurately, what you do with it. By capturing more of the data within your enterprise, augmenting it with additional data from your customers and other sources, pulling it all together (both historical and fresh), and examining it in new ways, you can extract the insights you need to deliver compelling digital experiences for every type of customer, across virtually every industry and scenario.

To get the most out of your data, you need a single, scale-out infrastructure on which you can capture and analyze a variety of data at unprecedented volumes and velocity. Specifically, you need to:

  • Build a highly scalable "big data warehouse" capable of ingesting and processing both structured and unstructured data. Your organization likely has most of the structured data within your walls, such as information on customers, products, and sales. However, you can gain new insights by augmenting that structured data with unstructured data from other, often external sources—such as weather predictions, social media content, media files, streaming IoT data, or application telemetry.
  • Apply sophisticated machine learning and advanced analytics to find the hidden patterns in your data and derive new insights. Then operationalize those insights via machine-learning models and scoring APIs, and push them back into a globally distributed database like Azure Cosmos DB. Ultimately, the data within your traditional stores is enriched with newly derived insights for real-time personalization and deeper customer engagement.
  • Leverage the same infrastructure that supports the above two (batch-oriented) processes to implement real-time processing—in other words, a Lambda architecture that supports both batch (cold) and stream (hot) processing methods. With this, your possibilities are endless. Detect fraud in real time. Use IoT telematics to predict which machines on an assembly line might be about to fail. Monitor real-time physiological telemetry to identify life-threatening conditions. You get the idea.

A comprehensive big-data and advanced analytics platform

With the big data and advanced analytics services in Azure, you have a tool set that enables you to do all the above. What's more, you can provision fast and scale even faster—paying for only what you use. You also benefit from granular security, automatic threat detection, market-leading SLAs, broad compliance certifications, a datacenter network with unprecedented worldwide coverage, and your choice of tools across both Microsoft and open-source services.

Figure 4. Microsoft Azure supports a single, scale-out architecture for capturing and analyzing a variety of data.

Big-data and data warehouse

You can use the following services to store and manage all types of data at hyper-scale:

  • For structured data, use Azure SQL Data Warehouse for elastic scalability with massively parallel processing.
  • For unstructured data, use Azure Data Lake Store to get massive throughput and analytic performance, with no file-size limits—or use Azure Blob storage if you're cost conscious and your needs are more modest.

Figure 5. Big data and data warehouse.

Analytics and data science

You can use the following services to find the hidden patterns in your data and derive new insights:

Figure 6. Analytics and data science.

Stream processing and information management

You can use the following services to orchestrate stream processing and information management:

  • Use Azure Data Factory to build pipelines and collect and orchestrate data for easier analysis.
  • Use Azure Event Hubs to provide a staging area for incoming streaming data.
  • Use Apache Kafka for HDInsight to buffer high-frequency, low-latency, real-time data.
  • Use Azure Functions to capture change feeds from underlying operational data stores (such as Azure Cosmos DB) using an event-driven, serverless architecture.

Figure 7. Stream processing and information management.

Customer case study: PROS

PROS offers a dynamic pricing service that analyzes thousands of internal and external factors to help customers keep their pricing accurate, competitive, and profitable. PROS achieves this by running enormously complex calculations on multiple terabytes of data. When the company made its move to the cloud, compatibility with Apache Spark, speed, scalability, and high availability were all essential requirements.

PROS chose Microsoft Azure and implemented Microsoft Azure HDInsight, which meant that the company could run its entire service platform on a single architecture. "Moving from on-premises to the cloud was essential, but we also hoped we could find a cloud system that offered us a complete computing platform," says Weiping Wang, Senior Software Architect at PROS. "This would free us up from managing the Spark cluster ourselves, which would become a much bigger job as we grew."

Apache Spark for Azure HDInsight offered the compatibility with Apache Spark technology that PROS required—including parallel processing, which multiplied the volume of data that could be processed in a given time. Apache Spark for Azure HDInsight also offered functionality specific to running large computations with vast amounts of data. Jobs could be set up easily—regardless of their size and in a fraction of the time previously required—by spinning up an instance of Azure HDInsight. "Rather than having to configure hardware to keep up with demand, Azure HDInsight allows us to keep up with demand dynamically," says Wang.

Azure HDInsight also gives PROS an industry-leading SLA, with 99.9 percent availability, to help ensure business continuity and help protect against catastrophic events. Plus, changing the company's compute platform to HDInsight has helped the company convert manual functionality into automation in its product, making it easier for customers to update their pricing models as their businesses evolve.

In HDInsight, a process that formerly took several days now takes just a few minutes. Ed Gonzalez, Product Manager, PROS

Read the entire PROS case study.

Hear from other customers who are building big-data and analytics solutions on Azure.

For more information

Scenario 5: AI

People don't want to adapt to technology. They want it to adapt to them—to help them "get things done" in a way that's easy and natural, and that complements how they perceive the world around them, process information, and interact with their surroundings. Recent advances in artificial intelligence (AI)—the computer simulation of intelligent human behavior—are making this possible to an extent that, only a few years ago, seemed like science fiction. But how can you harness the power of AI within your own apps, augmenting them with human characteristics such as speech, vision, language, and knowledge to make them even more compelling and useful?

At Microsoft, AI is all about amplifying human ingenuity with intelligent technology. That's why we're infusing AI into Cortana, our personal digital assistant; building AI into our database engines for greater performance and security; and equipping organizations with the power of AI through business solutions, accelerators, and practices.

It's also why we're building an AI platform with cutting-edge cognition technologies that empower developers to make their own apps more intelligent and delight their customers. For instance, you could use the Microsoft AI platform to build an "intelligent chatbot" that uses speech, vision, language, and knowledge to connect with users through your own apps or via Facebook Messenger, Cortana, Slack, Skype, and Bing.

The Microsoft AI platform: Bringing AI to every developer

You can use Microsoft Cognitive Services and the Microsoft Bot Framework, two components of the Microsoft AI platform, to add cognitive and conversational capabilities to your own apps—in a way that lets you reach out to users more naturally and contextually, wherever they are.

Cognitive Services

Microsoft Cognitive Services are a collection of powerful APIs that you can use to support natural and contextual interactions in your apps, enabling them to intelligently interpret the world and engage your users. The APIs are implemented as REST calls, so you can easily take advantage of them with just a few lines of code—knowing they work with any language, framework, or platform.

Microsoft Cognitive Services include the following categories:

  • Vision—Moderate content automatically and build more personalized apps by using state-ofthe-art image-processing algorithms that return smart insights about faces, images, and emotions.
  • Speech—Process spoken language in your applications. Hear and speak to your users by translating speech, filtering noise, identifying speakers, and understanding intent.
  • Language—Allow your apps to process natural language, evaluate sentiment and topics, translate text, and learn how to recognize what users want.
  • Knowledge—Map complex information and data to support semantic search or create a knowledge base of questions and answers. Tap into rich knowledge amassed from the web, academia, or your own data.
  • Search—Make your apps, webpages, and other experiences smarter and more engaging with Bing Search APIs, which provide access to billions of webpages, images, videos, and news articles.
  • Labs—Discover, try, and provide feedback on new Cognitive Services technologies before they're generally available in the marketplace.

Try the Cognitive Services APIs for free, in minutes.

Bot Framework

The Microsoft Bot Framework is a platform for building, connecting, testing, and deploying intelligent bots that can naturally interact with your users at scale, in a conversational way—for instance, to provide or capture information, make recommendations, or perform tasks.

With the Bot Framework, you get:

  • The open-source Bot Builder SDK, which supports .NET, Node.js, and REST, so you can get started building simple or sophisticated dialogues using the skills you already have.
  • Easy access to Cognitive Services, enabling your bot to see, hear, interpret, and interact in more human ways.
  • Connectors for reaching users wherever they are:
    • First-party (Microsoft) channels, including Bing, Cortana, Skype for Business, Microsoft Teams, Skype, Web Chat, Email, and GroupMe.
    • Third-party channels including Facebook, Kik, Slack, Telegram, and Twilio.
    • The Direct Line channel, which you can use to enable your own website or client application to communicate with your bot.
  • Azure Bot Service, which provides a way to easily host your bots using a hosting model based on either Azure Functions or Azure Web Apps.

Figure 8. The Microsoft AI platform makes it easy to build intelligent apps that connect with users more contextually and naturally, wherever they are.

Customer case study: Nedbank

Nedbank, a major bank in South Africa, wants to engage with their clients through the communication channels they already use, such as WhatsApp, Facebook Messenger, Slack, and other messaging apps. As a first step toward that goal, Nedbank built a proof-of-concept virtual call center solution based on Microsoft Cognitive Services and the Microsoft Bot Framework. The virtual call center agent recognizes the context of clients' questions and answers the 10 inquiries for which it's programmed at 10 percent of the cost of a live agent.

"With the Microsoft Bot Framework, Nedbank gets the power and flexibility it needs to make virtual agents a success," says Paul Cartmel, Managing Director at Atura, the technology provider that assisted with the project. "For example, [the virtual agent] can distinguish what callers want when they ask the same questions in different contexts. It was easy for us to get up to speed, and [the Bot Framework] remains easy to work with."

The bank's long-term goal is to set up Nedgroup Investments, the asset management division of Nedbank, for aggressive, planned growth without having to add staff.

The shift to serving individual investors is a priority for us in the next 12 months. Bot technology plays a big role in that. Steven Goodrich, Head of Technology, Nedbank Wealth Division

Read the entire Nedbank case study.

Hear from other customers who are using the Microsoft Bot Framework and Microsoft Cognitive Services.

For more information

Scenario 6: Security and compliance

No matter what you're building, you need to protect users' data and ensure compliance with any applicable regulations. You can do some of this within your app, which is why key security features that are specific to certain Azure services are covered earlier in this e-book, under the relevant scenarios for those services. However, when you rely on a cloud service provider for your infrastructure, you're also relying on that provider to help you keep your customers' data secure. It's the same with regulatory compliance; if the cloud services you're using aren't compliant, then neither is your app.

One of the best reasons to use Azure for your applications and services is to take advantage of its many built-in security tools and capabilities—additional layers of protection that you can use to achieve a defense-in-depth approach. For example, across all scenarios, you can use Azure Security Center for increased visibility and control over the security of all your Azure resources, use Azure Active Directory to help secure access to on-premises and cloud applications, use Azure Key Vault to safeguard cryptographic keys and other secrets used by your cloud apps and services, and use Azure Multi-Factor Authentication to provide even more security for your data and apps.

Microsoft Azure: The trusted platform

The Microsoft Azure infrastructure is designed from facility to applications for hosting millions of customers simultaneously, providing a trustworthy foundation upon which you can meet your needs. Security and privacy capabilities are built-in from the start, beginning with the Security Development Lifecycle (SDL), which addresses security at every development phase and ensures that Azure is continually updated to make it even more secure. Operational Security Assurance builds on SDL knowledge and processes to supply a framework that helps provide secure operations throughout the lifecycle of cloud-based services.

Functional security areas

Depending on the cloud service model, responsibility varies for managing security at different solution layers. Azure provides a wide array of configurable security options so you can customize security to meet your unique requirements.

Security capabilities built into the Azure platform cover all major functional areas, including network security, database security, storage security, compute security, operational security, security monitoring and management, and more. The introduction to Azure security provides an overview of all these areas.

Figure 9. Shared security responsibilities for different cloud service models.


In a world where data breaches and government requests for access to online customer information happen daily, you need a cloud platform that helps you ensure regulatory compliance. Microsoft provides the most comprehensive set of compliance offerings, including certifications and attestations, of any cloud service provider.

For example, in May 2018, a European privacy law, the General Data Protection Regulation (GDPR), is scheduled to take effect. The GDPR imposes new rules on companies, government agencies, nonprofits, and other organizations that offer goods and services to people in the European Union (EU), or that collect and analyze data tied to EU residents, and it applies no matter where you are located. Microsoft is committed to GDPR compliance across its cloud services when enforcement begins May 25, 2018, and provides GDPR related assurances in its contractual commitments.

Customer case study: Merck KGaA

Founded in 1668, Merck KGaA, Darmstadt, Germany, has grown to become one of today's leading healthcare, life sciences, and performance materials companies in the world. When the company first replaced several disparate internal platforms with EVA, an integrated digital workplace and social collaboration tool, employees connected to it and other internal applications via a virtual private network (VPN). As the company grew, routing all traffic through its on-premises environment became less and less tenable. And access via VPN didn't permit the use of multifactor authentication, which the company needed.

The company used Azure Active Directory Premium to give employees highly secure, single sign-on access to both the company's core cloud-based SaaS services and on-premises applications from their desktop computers or mobile devices. "By using Microsoft Azure, we are able to create a secure platform that allows us to embrace digital opportunities—SaaS applications and mobility device connectivity—that we knew would make our teams more productive," says Conor O'Halloran, Head of Identity Management for Merck KGaA, Darmstadt, Germany.

With Azure services, we have streamlined our identity environment and made it more secure by giving employees access to a platform we trust in the Microsoft cloud. Marc Autenrieth, Head of Core Solutions, Merck KGaA, Darmstadt, Germany

Read the entire Merck KGaA, Darmstadt, Germany case study.

For more information

Innovate faster with Azure data services

With its broad portfolio of data services, Microsoft Azure can help you build breakthrough new apps for any scenario, across your choice of stack, with innovative technology like AI built-in to wow your customers while making you more productive. The Microsoft Cloud supports more than a billion customers in more than 140 countries and regions, providing a unique platform that helps you achieve your business goals.

Azure supports a fully hybrid architecture, which helps you deliver the functionality you need regardless of where your data or compute power resides: your own datacenter, an Azure datacenter, other public cloud datacenters, or even a mobile device. A hybrid architecture also facilitates a phased approach. For instance, it helps you focus on harnessing the data within your walls before starting to tie in data from customers, suppliers, and other sources.

With Azure, you can:

  • Easily build and deploy anywhere. Use your existing skillsets and favorite tools to integrate data and build intelligent apps, and then deploy without a change in code. Build once and deploy anywhere—in the cloud, on-premises, and to edge devices—knowing that the global distribution of Azure means you can reach your users wherever they are.
  • Create an impact with an open platform. Maximize your effectiveness using the technology of your choice, including open source, backed by a platform that offers unparalleled performance, availability, and security. You can choose from several languages, employ a comprehensive range of data engines and processing technologies, and then deploy on your favorite platform.
  • Develop apps with built-in intelligence. Creating intelligent apps with Azure is easy; it includes advanced analytics and a rich set of cognitive APIs that provide human-like intelligence to enable more natural and contextual user interactions. No other platform brings analytics and native AI to your data wherever it lives, in the languages you use.

Wherever your data is, in whatever form it takes, Azure helps you unlock its full potential. Visit Azure data services to learn more and get started today.