Did you know that you can run R and Python code within a SQL Server instance? Not only does this give you a powerful server to process your data science calculations, but it makes things faster by eliminating the need to move data between client and server. Best of all, you can access this power via your favourite interface on your local machine, like the RStudio IDE for R or the VS Code Python extensions. 

To make this work, all you need is the necessary functions to launch computations in SQL Server, provided in the revoscaler package for R, and the revoscalepy library for Python. The video above demonstrates how to set things up for Python, and R users should check out the tutorials Learn in-database analytics using R in SQL Server.

Hotel booking

 

Second, Bing provides historical price trends for the date range you’re exploring to help make price-based trade-offs. Many sites only let you see the rate after you’ve already selected a date, so users end up clicking through many times to check the rates throughout the date range they’re interested in. Our price trend feature allows users to browse price trends over time in a single view.


 

Third, our new comparison view provides a comprehensive overview of pricing by hotel option. No more digging through multiple sites and reviews to find out what amenities are offered and if there are hidden fees! You’ll simply be able to see the detailed breakdowns side-by-side so you can feel assured you’re making the best choice for your needs and budget.


 

Home services pricing and scheduling

 

 

Coupons and deals

This comedy sketch from the New York Times imagines a possible new angle for a commercial from the Australian Tourist Commission. (NSFW language in the punchline at the very end.)

I've had a fantastic week this week hanging with my team in Redmond, and we'll be back with more from the blog next week. Have a great weekend!

Today, the healthcare industry is confronting many complex and daunting challenges that include demands to:

• Increase patient engagement.
• Take advantage of big data, analytics, artificial Intelligence (AI), and machine learning (ML).
• Integrate consumer health apps, wearables, and the Internet of Medical Things (IoMT).
• Combat cybersecurity threats, breaches, and ransomware.

In the midst of this, however, healthcare organizations must continue to:

• Deliver the best patient care.
• Improve patient outcomes.
• Reduce healthcare costs (now 7 percent of GDP in the UK and almost 18 percent of GDP in the United States).
• Enhance patient and clinician experiences.

And all with limited budget and resources!

Cloud computing can help healthcare organizations focus on patient care and reducing costs, and it enables IT to be more flexible, agile, scalable, and secure as the healthcare industry changes and grows.

A key challenge to adopting cloud computing is that healthcare needs solutions, not IT projects. Healthcare organizations of every size often have limited IT and cybersecurity resources burdened with maintaining existing IT infrastructure.

So how can they create new solutions?

Rx: Blueprints

To rapidly acquire new capabilities and implement new solutions, healthcare IT and developers can now take advantage of industry-specific Azure Blueprints. These are packages that include reference architectures, guidance, how-to guides and other documentation, and may also include executable code and sample test data built around a key use case of interest to healthcare organizations. Blueprints also contain components to support privacy, security, and compliance initiatives, including threat models, security controls, responsibility matrices, and compliance audit reports.

Here we focus on four new blueprints created to support UK National Health Service (NHS) cloud initiatives, as well as an existing blueprint for AI in healthcare that includes HIPAA and HITRUST compliance support. Any of these blueprints can be used worldwide with suitable customization.

Using these blueprints, your healthcare IT team can quickly implement a new type of solution in your secure Azure cloud. If this were a documentation-only package, your IT team would start from zero, with developers implementing 100 percent of the solution. With a blueprint, however, you get 50 percent to 90 percent of the end solution. Then, you can simply devote your efforts and resources to customizing the blueprint solution. The key to accelerating the development of your initiative is to minimize the scope by reducing the gap between the blueprint you pick and your target use case as illustrated in the diagram below:

This example illustrates customizing the data warehouse blueprint for a target use case such as a clinical data warehouse for a life sciences organization.

Develop a customized solution using an Azure Blueprint

1. If you haven’t already got an Azure environment, establish one. You can get started free in less than 10 minutes.
2. Pick the blueprint closest to your target use case.
3. Update the architecture if needed. Typically, the architecture for your target use case will remain the same as (or very similar to) the blueprint you choose.
4. Customize the blueprint to your target use case including updating documentation.
5. Switch any test data to your data.
6. Change any components as needed—for example, if you need a different database.
7. Update security and compliance to meet internal organizational requirements. This is based on your privacy and security risk assessment and compliance with all regulatory rules, security standards, and data protection law requirements.
8. Implement any other changes required to meet your business and technical requirements.

Take the next step: download an Azure Blueprint to accelerate your cloud initiative

UK NHS Data Analytics Blueprint

A reference architecture and guidance for a data analytics solution suitable for the collection, storage, analysis, and retrieval of sensitive healthcare data.

UK NHS Data Warehouse Blueprint

A reference architecture and guidance for a data warehouse solution suitable for securely ingesting, staging, storing, and interacting with sensitive healthcare data.

UK NHS IaaS Web Application Blueprint

A reference architecture and guidance for an infrastructure as a service (IaaS) web application suitable for the collection, storage, and retrieval of sensitive healthcare data.

UK NHS PaaS Web Application Blueprint

A reference architecture and guidance for a platform as a service (PaaS) solution suitable for the collection, storage, and retrieval of sensitive healthcare data.

Other Azure Blueprints

• HIPAA/HITRUST Health Data and AI

Collaboration

I post regularly about new developments in healthcare, cloud computing, security, privacy, compliance, blockchain, and AI on social media. You can find me on LinkedIn and Twitter.

What other opportunities and challenges do you see with blueprints for a secure healthcare cloud? I welcome your feedback and questions.

Cloud-hosted workloads offer excellent scalability, ease of deployment, and pre-secured infrastructure for your workloads. However, the workloads themselves may still be susceptible to attack by cybercriminals. To help safeguard your resources in the cloud, you need to be able to keep up with threats, harden your resources that could be vulnerable to attacks, and deploy techniques to ensure that protection mechanisms are working.

Avyan Consulting partnered with the Azure Security Center team to build attack simulation playbooks for demonstration and training purposes. Azure administrators may use these playbooks to deploy fully operational web and Compute workloads, security management tools such as Azure Security Center & Web App Firewalls (WAFs), and SQL threat protection. Once deployed, the administrator can invoke attacks against the workloads, executing similar techniques used by adversaries around the world. These attack simulations are applied using the supplied instructions and automation. The playbooks cover four common attack scenarios simulations:

This experience will enable Azure customers to learn about hardening best practices, defensive configurations, and what to look for should a real attack occur.

If you are responsible for the security of datacenter workloads running in the cloud, learn more about Azure’s native security tools such as  Azure Security Center, SQL Threat Detection, and Application Gateway (WAF). These tools work with Cloudneeti from Avyan Consulting to provide rich compliance views across Azure resources.

Now in preview

Announcing public preview of Azure IoT Hub manual failover feature - IoT Hub cloud service manual failover enables customers to failover an IoT hub instance from its primary Azure region to its corresponding geo-paired region with a recovery time objective (RTO) of 10 min–2 hours. The IoT Hub service provides cross-regional automatic disaster recovery as a default mitigation for region-wide failures or extenuated outages, which has an RTO of 2 – 26 hours. If your business uptime goals are not satisfied by the RTO that the Microsoft-initiated failover provides, you should consider using manual failover to trigger the failover process yourself.

Azure Monitor: Route AAD Activity Logs using diagnostic settings - Azure Monitor diagnostic settings enable you to stream log data from an Azure service to three destinations: an Azure storage account, an Event Hubs namespace, and/or a Log Analytics workspace. This allows you to easily route logs from any Azure service to a data archive, SIEM tool, or custom log processing tool. Now you can route your AAD audit and sign-in logs to these same destinations, centralizing all of your Azure service logs in one pipeline.

Now generally available

General availability of instance size flexibility for Azure Reserved Virtual Machine Instances - Instance size flexibility for Azure Reserved Virtual Machine Instances is a new feature that makes your reserved instance purchasing and management even simpler by applying reservation discounts to different virtual machine (VM) sizes within the same VM group. With instance size flexibility, you don’t have to deploy the exact same VM size to get the benefit of your purchased Azure Reserved Instances (RI) as other VM sizes within the same VM group also get the RI discount.

Azure management groups now in general availability - Azure management groups enable you to organize your subscriptions and apply governance controls, such as Azure Policy and Role-Based Access Controls (RBAC), to the management groups. All subscriptions within a management group automatically inherit the controls applied to the management group. No matter if you have an Enterprise Agreement, Certified Solution Partner, Pay-As-You-Go, or any other type of subscription, this service gives all Azure customers enterprise-grade management at a large scale for no additional cost. Management groups not only allow you to group subscriptions but also allows you to group other management groups to form a hierarchy.

Announcing Linux on App Service Environment General Availability - Linux on App Service Environment combines the features from App Service on Linux and App Service Environment. Linux customers can take advantage of deploying Linux and containerized apps in an App Service Environment, which is ideal for deploying applications into a VNet for secure network access or apps running at a high scale.

Also generally available

• General availability: Start/Stop VMs during off-hours in Azure Automation

News and updates

Ansible 2.6: What’s new for Azure - Azure in Ansible 2.6 includes four new Azure modules in addition to over 17 enhancements to existing modules that enable a greater range of scenarios for the deployment and management of virtual machines and networking objects. Microsoft and Red Hat engineering teams continue to develop the Azure modules with the Ansible community.

Automatic intelligent insights to optimize performance with SQL Data Warehouse - Azure SQL Data Warehouse (SQL DW) now provides intelligent performance insights within the Azure portal. Data skew and suboptimal table statistics are common issues that can degrade the performance of your data warehouse if left unchecked. At no additional cost, SQL DW surfaces intelligent insights for all Gen2 data warehouses and is tightly integrated with Azure Advisor to deliver you best practice recommendations.

Managed Disks migration now available in the Azure Portal - Convert VMs with unmanaged Disks to Managed Disks in the Azure Portal without requiring PowerShell or CLI scripts. Azure Managed Disks simplifies disk management for Azure IaaS VMs by managing the storage accounts associated with the VM disks. You only have to specify the type (Standard HDD, Standard SSD, or Premium SSD) and the size of disk you need, and Azure creates and manages the disk for you.

Additional news and updates

• Azure Data Factory/Azure Cosmos DB connector now supports UPSERT
• Azure API Management update August 3
• PHP minor version update for September 2018
• System Center Operations Manager Management Pack to configure Operations Management Suite
• Power BI Embedded available in Azure China cloud
• Power BI Embedded API for programmatic control of visual headers
• Log Analytics workspaces must be created in the Azure portal
• Azure Site Recovery Update Rollup 27
• OMS Linux Agent July 2018 release now available

The Azure Podcast

Technical content and training

Experts tips on hardening security with Azure security - Learn how the Microsoft Threat Intelligence Center is helping to secure Azure and the global security landscape. The team also has a broad view across many geographies and a view of the services that run in Azure. With this insight, the team can see common attack patterns. These patterns can be at the network level, service level, app level, or OS level. As soon as an exploit is detected, the Microsoft Threat Intelligence Center works with other teams at Microsoft to build mitigations into our products and services.

Getting started with IoT: what do you do with all that data? - Once your IoT devices are deployed, secured, and provisioned through Azure IoT Hub, the question remains: where do you send all of the data? The role of Azure IoT Hub is to determine how each data packet needs to be prioritized and where to send it. Knowing how to handle the information you generate, and where to route it, can be a challenge; however, once you start establishing connections, you can begin to comprehend the potential of IoT to transform your business.

Real example: improve accuracy, reduce training times for existing R codebase - Retailers have been building product recommendation systems for years, and many of those are built using the programming language R. For older implementations of recommender systems, it’s time to consider improving performance and scalability by moving these systems to Azure. Learn the process of successfully optimizing and reusing an existing recommendation system written in R using the parallelism of the MicrosoftML and RevoScaleR libraries built into Microsoft Machine Learning Server.

Azure Friday

Events

Microsoft Azure Data welcomes Data Platform Summit attendees - Data Platform Summit 2018 is taking place in Bangalore this week. Data Platform Summit 2018 is the only Data Platform Conference in Asia focusing on Microsoft Data Platform and Open Source. Rohan Kumar, Corporate Vice President, Azure Data will be there to deliver the keynote and share the work happening on the Azure Data team at Microsoft. In this post, Rohan covers the key trends that are defining and shaping the evolution of data management, and how Microsoft delivers a consistent and global platform optimized for hybrid cloud for all applications and data.

Azure tips & tricks

Customers and partners

Responsibilities of a partner/system integrator in managing Azure Cosmos DB - Many customers are using Azure Cosmos DB all around the world. This article lists the actions a partner can perform in different areas of Azure Cosmos DB such as security, performance management, and more.

Azure Marketplace consulting offers: April - Earlier this year, the Azure Marketplace ecosystem added consulting offers to the growing lineup of software providers. In April, we published 73 new offers that met the onboarding criteria.

The IoT Show

Industries

Financial Services | How to upgrade your financial analysis capabilities with Azure - To assess risk, analysts review research, monitor economic and social conditions, stay informed of regulations, and create models for the investment climate. In short, the inputs into an analysis make for a highly complex and dynamic calculation, one that requires enormous computing power. Get a technical overview of using Microsoft Azure to support and enhance risk grid computing in banking, including recommended systems and high-level architectures.

Retail | Azure cloud business value for retail and consumer goods explained - Cloud technologies are enabling new capabilities and those new powers are disrupting the business models of traditional retailers and sellers of consumer goods. The cloud is at the heart of digital transformation. It is no longer a question of “if,” but “when” and “how” to move to the cloud for most brands. To move an e-commerce solution to the cloud is a significant task, with costs that must be understood by a decision maker. Take a look at a path a retail business can take to migrate its existing infrastructure to Azure.

Health and life sciences | Improve collaborative care and clinical data sharing with blockchain - Blockchain offers new capabilities to greatly improve health information exchange. At Microsoft we are working to maximize the benefits of solutions that have the potential to improve patient outcomes, reduce healthcare costs, and enhance the experiences of patients and healthcare workers. Learn about a new partner solution and pilot for a better health information exchange that uses blockchain, and runs in Microsoft Azure.

A Cloud Guru - Azure This Week

Typically, calendar eras represent long time periods. In the Gregorian calendar, for example, the current era spans (as of this year) 2,018 years. In the Japanese calendar, however, a new era begins with the reign of a new emperor. On April 30, 2019, Emperor Akihito is expected to abdicate, which will bring to an end the Heisei era. On the following day, when his successor becomes emperor, a new era in the Japanese calendar will begin. It is the first transition from one era to another in the history of .NET, and the first change of eras in the Japanese calendar since Emperor Akihito’s accession in January 1989. In this blog post, I’ll discuss how eras work in general in .NET, how you can determine whether your application is affected by the era change, and what you as a developer have to doto make sure your application handles the upcoming Japanese era changes successfully.

Calendars in .NET

.NET supports a number of calendar classes, all of which are derived from the base Calendar class. Calendars can be used in either of two ways in .NET. A supported calendar is a calendar that can be used by a specific culture and that defines the formatting of dates and times for that culture. One supported calendar is the default calendar of a particular culture; it is automatically used as that culture’s calendar for culture-aware operations. Standalone calendars are used apart from a specific culture by calling members of that Calendar class directly. All calendars can be used as standalone calendars. Not all calendars can be used as supported calendars, however.

Each CultureInfo object, which represents a particular culture, has a default calendar, defined by its Calendar property. The OptionalCalendars property defines the set of calendars supported by the culture. Any member of this collection can become the current calendar for the culture by assigning it to the CultureInfo.DateTimeFormat.Calendar property.

Each calendar has a minimum supported date and a maximum supported date. The calendar classes also support eras, which divide the overall time interval supported by the calendar into two or more more periods. Most .NET calendars support a single era. The DateTime constructors that create a date using a specific calendar assume that that dates belong to the current era. You can instantiate a date in an era other than the current era by calling an overload of the Calendar.ToDateTime method.

The JapaneseCalendar and JapaneseLunisolarCalendar classes

Two calendar classes, JapaneseCalendar and JapaneseLunisolarCalendar, are affected by the introduction of a new Japanese era. These calendars use the same months and days of the month as the Gregorian calendar. They differ in how they calculate years; the reign of a new emperor marks the beginning of a new era, which begins with year 1.

The JapaneseCalendar and JapaneseLunisolarCalendar are the only two calendar classes in .NET that recognize more than one era. Neither is the default calendar of any culture. The JapaneseCalendar class is an optional calendar supported by the Japanese-Japan (ja-JP) culture and is used in some official and cultural contexts. The JapaneseLunisolarCalendar class is a standalone calendar; it cannot be the current calendar of any culture. That neither is the default calendar of the ja-JP culture minimizes the impact that results from the introduction of a new Japanese calendar era. The introduction of a new era in the Japanese calendar affects only:

• Culture-specific date and time operations with cultures that support the JapaneseCalendar, when the JapaneseCalendar is the current calendar for the culture.
• Calls to standalone JapaneseCalendar and JapaneseLunisolarCalendar methods, such as GetDaysInYear, IsLeapYear, or ToDateTime.
• Formatting using the “g” or “gg” custom format specifier.
• Year formatting with the Japanese calendar classes, which will change to provide special formatting for the first year of an era. For more information, see The first year of an era later in this blog post.

Note that, with the exception of the “g” or “gg” custom format specifier, any unintended side effects from the change in Japanese eras occur only if you use a Japanese calendar class as a standalone calendar, or if you use the JapaneseCalendar as the current calendar of the ja-JP culture.

Testing era changes on Windows

The best way to determine whether your applications are affected by the new era is to test them in advance with the new era in place. You can do this immediately for .NET Framework 4.x apps and for .NET Core apps running on Windows systems. For .NET Core apps on other platforms, you’ll have to wait until the ICU globalization library is updated; see the Updating data sources section for more information.

For .NET Framework 4.x apps and for .NET Core apps on Windows systems, era information for the Japanese calendars is stored as a set of REG_SZ values in the HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlNlsCalendarsJapaneseEras key of the system registry. For example, the following figure shows the definition of the Heisei era in the Japanese calendar.

You can use the Registry Editor (regedit.exe) to add a definition for the new era. The name defines the era start date, and its value defines the native era name, native abbreviated era name, English era name, and English abbreviated era name, as follows:

Name: yyyy mm dd Value: <native-full>_<native-abbreviated>_<English-full>_<English-abbreviated>

Since the new era name has not been announced, you can use question marks as a placeholder. For the native full and abbreviated name, you can use the FULLWIDTH QUESTION MARK (U+FF1F), and for the English full and abbreviated name, you can use the QUESTION MARK (U+003F). For example:

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlNlsCalendarsJapaneseEras]
"2019 05 01"="？？_？_??????_?"

Once the new era information is in place on any system running .NET, you can use code like the following to identify instances in which the current string representation of a date and time in a Japanese calendar will differ from its string representation after the introduction of the new era:

Note that the new era will begin on May 1, 2019, and the Japanese government is expected to announce the official name of the new era on about April 1, 2019. A window of approximately one month leaves very little time to test, detect bugs, troubleshoot, and address bugs. It is important that applications be adequately tested well in advance of the beginning of the new era.

.NET changes to support the new era

To ensure that the transition to a new Japanese calendar era is as seamless as possible, the following changes will be introduced in .NET before the formal introduction of the new Japanese calendar era. These changes will be released in the near future.

Updating data sources

Currently, the way in which calendar era information is stored differs across .NET implementations:

• For .NET Framework 4.0 and later, as well as for .NET Core running on Windows, calendar era information is provided by the Windows operating system and retrieved from the system registry.An update to Windows will add the new era value to the registry once the era name and abbreviated era name are known. .NET on Windows will automatically reflect this update.
• For .NET Framework 3.5 on Windows, calendar era information is maintained as hard-coded data by the .NET Framework itself.An update to .NET Framework 3.5 will change its source for calendar data from private hard-coded data to the registry. Once this happens, .NET Framework 3.5 will automatically reflect the eras defined in the Windows registry.
• For .NET Core on non-Windows platforms, calendar information is provided by International Components for Unicode (ICU), an open source globalization library.ICU libraries will be updated once the era name and abbreviated era name are known.
  Because they do not depend on ICU cultural data, applications that use the globalization invariant mode are not affected by this change.

Updates will be released as soon as possible after the new era name is announced.

Relaxed era range checks

In the past, date and time methods that depend on calendar eras threw an ArgumentOutOfRangeException when a date and time was outside the range of a specified era. The following example attempts to instantiate a date in the 65th year of the Showa era, which began on December 25, 1926 and ended on January 7, 1989. This date corresponds to January 9, 1990, which is outside the range of the Showa era in the JapaneseCalendar. As a result, an ArgumentOutOfRangeException results.

To accommodate the era change, .NET by default will use relaxed range enforcement rules. A date in a particular era can “overflow” into the following era, and no exception is thrown. The following example instantiates a date in the third quarter of year 31 of the Heisei era, which is more than two months after the Heisei era has ended.

If this behavior is undesirable, you can restore strict era range checks as follows:

• .NET Framework 4.6 or later: You can set the following AppContextSwitchOverrides element switch:
  
• .NET Framework 4.5.2 or earlier: You can set the following registry value:
  

  Key	HKEY_LOCALMACHINESOFTWAREMicrosoft.NETFrameworkAppContext
  Name	Switch.System.Globalization.EnforceJapaneseEraYearRanges
  Type	REG_SZ
  Value	1

• .NET Core: You can add the following to the .netcore.runtimeconfig.json config file:
  

The first year of an era

Traditionally, the first year of a new Japanese calendar era is called Gannen (元年). For example, instead of Heisei 1, the first year of the Heisei era can be described as Heisei Gannen.

As part of its enhanced support for Japanese calendar eras, .NET will adopt this convention. However, parsing operations will successfully handle strings that include “1” or “Gannen” as the year component.

The following example displays a date in the first year of the Heisei era. The output from the example illustrates the difference between the current and future handling of the first year of an era by .NET. As the output from the example illustrates, the .NET formatting routine converts year 1 to Gannen only if the year is followed by the 年 symbol.

Handling Japanese calendar eras effectively

The change in Japanese eras poses a number of issues. The following list addresses some of these and proposes workarounds.

Specify an era when instantiating dates

You can instantiate a date using the date values of the Japanese calendar in any of three ways:

• By calling a DateTime constructor that includes a parameter of type Calendar.
• By calling a DateTimeOffset constructor that includes a parameter of type Calendar.
• By calling the Calendar.ToDateTime method of a specific calendar type.

The .NET calendar classes include a CurrentEra property, which indicates the current (or default) era used in interpreting dates expressed in a specific calendar. Its value is the constant 0. It is an index into the Eras property, which orders eras in reverse chronological order. In other words, the most recent era is always the default era.

When eras can change unexpectedly, calling a date and time instantiation method that relies on the default era can produce an ambiguous date. In the next example, the call to the JapaneseCalendar.ToDateTime method that uses the default era returns different dates depending on whether or not the new era has been defined in the registry. Note that the output for this and the following example uses the sortable date/time pattern.

You can do either of two things to avoid potential ambiguity:

• Call the Calendar.ToDateTime overload that specifies an era. This overload always produces an unambiguous date:
  

• Instantiate dates in the Gregorian calendar. Use the Japanese calendar or the Japanese Lunisolar calendar only for the string representation of dates.

Use relaxed era range checks

A basic problem of calendars that can add new eras is that you can’t be certain that a future date will always belong to the current era. If strict range checking is enabled, future dates that are valid before an era change may become invalid after the era change. For this reason, leave relaxed era range checking on (the default value).

Use the era in formatting and parsing operations

Because dates can be ambiguous, you should always format a date value with its era. This is the default behavior of the standard date and time format strings. If you are using a custom date and time format string, be sure to include the “g” or “gg” custom format specifier. Conventionally, the era precedes the other date components in the string representation of a Japanese calendar date.

For parsing operations, also ensure that an era is present unless you want all dates and times to default to the current era.

A call to action

The introduction of a new era in the Japanese calendar poses challenges for any application that uses either the JapaneseCalendar or the JapaneseLunisolarCalendar. We’ve discussed how eras work with calendars and dates and times in .NET, how .NET applications will be updated to use the new era, how .NET APIs are changing to help you handle the Japanese era change, and what you can do as a developer to test your application and minimize the effect of future era changes. Above all, we recommend that you:

• Determine whether your applications are affected by the Japanese era change. All applications that use the JapaneseCalendar and the JapaneseLunisolarCalendar classes may be affected.
• Test your application to determine whether it can handle all dates, and particularly dates that exceed the range of the current Japanese calendar era.
• Adopt the practices outlined in the Handling Japanese calendar eras effectively section to ensure that you can handle era changes effectively.

See also

The Japanese Calendar’s Y2K Moment
Testing for New Japanese Era
Japanese calendar
Japanese era name
List of Japanese era names
Working with Calendars

Microsoft Ignite is known as the place for technologists to build new skills, expand proficiencies, evaluate new tools, and network with the community. This year, we’re adding even more developer-focused content than ever before by going deep on popular and emerging developer technologies. But don’t just take my word for it, hear what Scott Hanselman has to say about this year's Ignite:

Join us in Orlando, September 24-28 to learn about the latest Microsoft innovations and advancements in cloud, client, and web development. Expand your skills through sessions, workshops, community activities, and challenge-based hack time. Dive deep with technical experts who want to hear your feedback and learn how our products can better support your development goals and use cases.

Register today to join us at Microsoft Ignite 2018

You'll get the opportunity to expand your knowledge in growth areas, such as IoT, AI, ML, DevOps, serverless, microservices, and containers. Take advantage of the languages and frameworks you already know, such as .NET, Java, and Node.js to expand your skills into these areas. Here are just a few examples of the sessions you’ll find this year at Ignite:

• .NET Overview and Roadmap - Learn about the present and future of Microsoft .NET Platform. We’ll look at the current state of .NET, its incredible growth, and new core features. We’ll also show how .NET will help you build cutting-edge solutions for the future.
• Azure Services Fabric Overview and the Road Ahead - Service Fabric powers Azure core infrastructure and large-scale services. Dive into the latest capabilities, including Service Fabric Mesh, containers, low-latency data processing, .NET Core 2.0, and Visual Studio 2017 integration. Learn how we’re making containerized microservice applications much easier to build.
• Fundamentals of Internet of Things (IoT) with Microsoft - See Microsoft’s IoT portfolio and where it’s headed. Discover how both Microsoft Azure and Azure Stack can deliver a hybrid environment for IoT success. Learn about IoT, product roadmap, and a variety of industry-based solutions to get you up and running with IoT quickly.
• Build Full-Stack Enterprise Node.js Apps with Visual Studio Code and Microsoft Azure - Node.js is a scalable JavaScript server runtime with a huge ecosystem of open source packages and a variety of front-end frameworks. Learn how Visual Studio Code, Microsoft’s open source, cross-platform editor, supercharges your productivity, including debugging front-end and server-side code, deploying, and even debugging remotely.

Microsoft Ignite also offers several immersive, Pre-Day sessions that will cover topics such as AI and building serverless applications in greater depth. Convinced? Register today to secure your choice of hotel and Pre-Day sessions—they’re filling up fast. I hope we'll see you in Orlando!

We’ve had great traction with our support of Ethereum on Azure. The existing Proof-of-Work solution has been deployed tens of thousands of times across a variety of industry verticals. Through the extensive development on our platform, we’ve received great feedback from the community that has helped us shape our next Ethereum ledger product. I’m excited to announce the release of Ethereum Proof-of-Authority on Azure.

Proof-of-Authority

Proof-of-Work is a Sybil-resistance mechanism that leverages computation costs to self-regulate the network and allow fair participation. This works great in anonymous, open networks where competition for cryptocurrency promotes security on the network. However, in private/consortium networks the underlying ether has no value. An alternative protocol, Proof-of-Authority, is more suitable for permissioned networks where all consensus participants are known and reputable. Without the need for mining, Proof-of-Authority is more efficient while still retaining Byzantine fault tolerance.

Enterprise-ready

We’ve built this solution with the same principles that we drive in all our production services at Microsoft. In Proof-of-Authority, each consensus node on the network has its own Ethereum identity. In the case that a node goes down, it’s important that the member doesn’t lose consensus participation. Ideally, each member would run redundant consensus nodes to ensure a highly available network presence. To accomplish this, we’ve built an abstraction which allows each consensus participant to delegate multiple nodes to run on their behalf. Each Azure Proof-of-Authority network comes with our identity leasing system that ensures that no two nodes carry the same identity. In the case of a VM or regional outage, new nodes can quickly spin up and resume the previous nodes’ identities.

Web Assembly Smart Contracts

Enterprise developers often cite Solidity language as one of the largest pain points when developing on Ethereum. To address this, we’ve enabled Parity’s web-assembly support, allowing developers to author smart contracts in familiar languages such as C, C++, and Rust.

Azure Monitor

This solution also comes with Azure Monitor to track node and network statistics. For application developers, this provides visibility into the underlying blockchain to track block generation statistics. Network operators can use Azure Monitor to quickly detect and prevent network outages through infrastructure statistics and queryable logs.

Extensible governance

Many of our customers want to participate in a consortium, but don’t want to manage the network infrastructure. We’ve leveraged Parity’s highly extensible Proof-of-Authority client to build a level of abstraction that allows our users to separate consortium governance from network operation. Each consortium member has the power to govern the network and can optionally delegate the consensus participation to the operator of their choosing. The Proof-of-Authority deployment comes with a Governance DApp to simplify voting and validator delegation. With this solution, each consortium member has custody over his or her own keys, allowing secure signing to be performed in the wallet of preference, for example, MetaMask in-browser wallet, Ledger hardware wallet, or Azure Key Vault with ECC signing.

Governance DApp features

• Decentralized governance - Changes in network authorities are administered through on-chain voting by select administrators.
• Validator delegation - Authorities can manage their validator nodes that are setup in each PoA deployment.
• Auditable change history - Each change is recorded on the blockchain, providing transparency and auditability.

Figure 1: Governance DApp allows for on-chain consortium management.

We’re excited to see the new applications that our customers build with Ethereum Proof-of-Authority in Azure. Check out our deployment guide to get started and learn more about the architecture and consortium governance.

Customers use HDInsight Interactive Query (also called Hive LLAP, or Low Latency Analytical Processing) to query data stored in Azure storage & Azure Data Lake Storage in super-fast manner. Interactive query makes it easy for developers and data scientist to work with the big data using BI tools they love the most. HDInsight Interactive Query supports several tools to access big data in easy fashion. In this blog we have listed most popular tools used by our customers:

Microsoft Power BI

Microsoft Power BI Desktop has a native connector to perform direct query against  HDInsight Interactive Query cluster. You can explore and visualize the data in interactive manner. To learn more see Visualize Interactive Query Hive data with Power BI in Azure HDInsight and Visualize big data with Power BI in Azure HDInsight.

Apache Zeppelin

Apache Zeppelin interpreter concept allows any language/data-processing-backend to be plugged into Zeppelin. You can access Interactive Query from Apache Zeppelin using a JDBC interpreter. To learn more please see Use Zeppelin to run Hive queries in Azure HDInsight.

Visual Studio Code

With HDInsight Tools for VS Code, you can submit interactive queries as well at look at job information in HDInsight interactive query clusters. To learn more please see Use Visual Studio Code for Hive, LLAP or pySpark.

Visual Studio

Visual Studio integration helps you create and query tables in visual fashion. You can create a Hive tables on top of data stored in Azure Data Lake Storage or Azure Storage. To learn more please see Connect to Azure HDInsight and run Hive queries using Data Lake Tools for Visual Studio.

Ambari Hive View

Hive View is designed to help you author, optimize, and execute queries. With Hive Views you can:

• Browse databases.
• Write queries or browse query results in full-screen mode, which can be particularly helpful with complex queries or large query results.
• Manage query execution jobs and history.
• View existing databases, tables, and their statistics.
• Create/upload tables and export table DDL to source control.
• View visual explain plans to learn more about query plan.

To learn more please see Use Hive View with Hadoop in Azure HDInsight.

Beeline

Beeline is a Hive client that is included on the head nodes of HDInsight cluster. Beeline uses JDBC to connect to HiveServer2, a service hosted on HDInsight cluster. You can also use Beeline to access Hive on HDInsight remotely over the internet. To learn more please see Use Hive with Hadoop in HDInsight with Beeline.

Hive ODBC

Open Database Connectivity (ODBC) API, a standard for the Hive database management system, enables ODBC compliant applications to interact seamlessly with Hive through a standard interface. Learn more about how HDInsight publishes HDInsight Hive ODBC driver.

Tableau

Tableau is very popular data visualization tool. Customers can build visualizations by connecting Tableau with HDInsight interactive Query.

Apache DBeaver

Apache DBeaver is SQL client and a database administration tool. It is free and open-source (ASL). DBeaver use JDBC API to connect with SQL based databases. To learn more, see How to use DBeaver with Azure #HDInsight.

Excel

Microsoft Excel is the most popular data analysis tool and connecting it with big data is even more interesting for our customers. Azure HDInsight Interactive Query cluster can be integrated with Excel with ODBC connectivity.To learn more, see Connect Excel to Hadoop in Azure HDInsight with the Microsoft Hive ODBC driver.

Try HDInsight now

We hope you will take full advantage fast query capabilities of HDInsight Interactive Query using your favorite tools. We are excited to see what you will build with Azure HDInsight. Read this developer guide and follow the quick start guide to learn more about implementing these pipelines and architectures on Azure HDInsight. Stay up-to-date on the latest Azure HDInsight news and features by following us on Twitter #HDInsight and @AzureHDInsight. For questions and feedback, please reach out to AskHDInsight@microsoft.com.

About HDInsight

Azure HDInsight is Microsoft’s premium managed offering for running open source workloads on Azure. Azure HDInsight powers mission critical applications ranging in a wide variety of sectors including, manufacturing, retail education, nonprofit, government, healthcare, media, banking, telecommunication, insurance, and many more industries ranging in use cases from ETL to Data Warehousing, from Machine Learning to IoT, and more.

Additional resources

• Get started with HDInsight Interactive Query Cluster in Azure.
• Zero ETL analytics with LLAP in Azure HDInsight.
• Azure HDInsight Performance Benchmarking: Interactive Query, Spark and Presto.
• Learn more about Azure HDInsight.
• Use Hive on HDInsight.
• Open Source component guide on HDInsight.
• HDInsight release notes.
• Ask HDInsight questions on Msdn forums.
• Ask HDInsight questions on stackoverflow.

Selecting how to monetize your app is a big part of any development process. Different kinds of apps can have very different opportunities and needs—so your monetization strategy can inform not just your business model, but your app’s final design, content, and code.

Microsoft Store offers a variety of different ways to add monetization to your app. Here’s a quick breakdown of our 4 biggest earners and how they can work for you.

Base price

Charging an up-front base price is a one-time transaction: you offer a product and the user pays for it. Microsoft Store allows you to change your base price at any time, in any market, so you can always experiment with your numbers and see how your target audience responds.

A general rule of thumb for setting a base price is to scope out the competition. Once you know what the market has paid in the past, you can set a price point for your own unique product.

You can see from the data below that charging a base price is the most used option in the Microsoft Store for apps.

In-app purchases

In-app purchases are microtransactions that allow customers to purchase additional content within an app itself. These small fees can build up to big returns—a Gartner Research report revealed that customers will spend 24% more on in-app purchases than they would to buy an app outright. Once a customer sees value, they have an incentive to pay.

Content is king for in-app purchases. The more your customers use or enjoy your app, the more likely they are to purchase new content that is bigger, better, or simply more.

In-app purchase is the most chosen method to monetize for games within the Microsoft Store, as shown below.

Subscriptions

With subscriptions, customers pay a recurring, scheduled fee to access your app. You select the amount, the schedule, and if it covers your entire app or applies to digital products within your app, like features or content.

The subscription model is ideal for apps that offer continuous content and services. In addition to a reliable revenue stream, you also gain access to a quantifiable base of loyal users.

Display mobile ads

Mobile advertising is one of most profitable app revenue models for developers with native, banner, and interstitial formats heading the list.

If you elect the ad route, include your designers so they can fold your advertising formats seamlessly into your app experience. You can find test ads and more information on best practices here.

And finally…

No matter which monetization strategy you choose, be sure to check out how you get paid. Comment below on which monetization strategy you’re using!

The post What’s your monetization strategy? appeared first on Windows Developer Blog.

Python retains its top spot in the fifth annual IEEE Spectrum top programming language rankings, and also gains a designation as an "embedded language". Data science language R remains the only domain-specific slot in the top 10 (where it as listed as an "enterprise language") and drops one place compared to its 2017 ranking to take the #7 spot.

Looking at other data-oriented languages, Matlab as at #11 (up 3 places), SQL is at #24 (down 1), Julia at #32 (down 1) and SAS at #40 (down 3). Click the screenshot below for an interactive version of the chart where you can also explore the top 50 rankings.

The IEEE Spectrum rankings are based on search, social media, and job listing trends, GitHub repositories, and mentions in journal articles. You can find details on the ranking methodology here, and discussion of the trends behind the 2018 rankings at the link below.

IEEE Spectrum: The 2018 Top Programming Languages

A few months ago we released ML.NET 0.1 at //Build 2018., ML.NET is a cross-platform, open source machine learning framework for .NET developers. We’ve gotten great feedback so far and would like to thank the community for your engagement as we continue to develop ML.NET together in the open.

We are happy to announce the latest version: ML.NET 0.4. In this release we’ve improved support for natural language processing (NLP) scenarios by adding the Word Embedding Transform, improved the speed of linear learners like binary classification and linear regression by adding support for the SymSGD learner, made improvements to the F# API and samples for ML.NET, bug fixes and more.

Additionally, we really want your feedback on making ML.NET really easy to use. We are working on a new API which improves flexibility and ease of use. When the new API is ready and good enough, we plan to deprecate the current “pipeline” API. Because this will be a significant change we want to share our proposals for the multiple API options and comparisons in a future blog post and start an open discussion with you where you can provide your feedback and help shape the long-term API for ML.NET.

The blog post below provides more details about the additions in the 0.4 release.

• Word Embedding Transform for Text Scenarios
• SymSGD Learner for Binary Classification
• Improvements to F# API and samples for ML.NET

Word Embeddings Transform for Text Scenarios

Word embeddings is a technique for mapping words to numeric vectors that are intended to capture some of the meaning of the words, so they can be used for visualization or model training.

The word embedding transform added to ML.NET enables using pretrained word embedding models in pipelines. “Pretrained” means you can use existing embeddings instead of needing to create your own (which takes a lot of data and time).  Several different pretrained models are available (GloVe, fastText, and SSWE).

By adding this transform in addition to existing transforms for working with text (like the TextFeaturizer), you can improve the model’s metrics.

For example, we can improve the accuracy of the sentiment analysis sample by 5% if we change the line with TextFeaturizer to:

// Change TextFeaturizer to output tokens (list of words in the text)
pipeline.Add(new TextFeaturizer("FeaturesA", "SentimentText") { OutputTokens = true});

// Add word embeddings 
pipeline.Add(new WordEmbeddings(("Features_TransformedText", "FeaturesB"))); 

// Combine the features from word embeddings and text featurizer into one column 
pipeline.Add(new ColumnConcatenator("Features", "FeaturesA", "FeaturesB"));

In the above example, we used the default word embeddings (SSWE: Sentiment-Specific Word Embeddings) which are helpful in sentiment tasks.

SymSGD Learner for Binary Classification

SymSGD is a parallel SGD algorithm that retains the sequential semantics of SGD but offers a much better performance based on multithreading. SymSGD is fast, scales well on multiple cores, while achieving the same accuracy as sequential SGD. It is now available in ML.NET for binary classification.

A related learner, Stochastic Gradient Descent (SGD) is a well-known and effective method for many machine learning problems such as regression and classification tasks. However, its performance scalability is severely limited by its inherently sequential computation.

SymSGD approach is applicable to any linear learner whose update rule is linear, such as binary classification and a linear regression.

Here’s how you add a SymSGD Binary Classifier learner to the pipeline:

pipeline.Add(new SymSgdBinaryClassifier() { NumberOfThreads = 1});


For additional sample code using SymSGD, check here.

The current implementation in ML.NET does not have multi-threading enabled, the issue is tracked by #655, but SymSGD can still be helpful in scenarios where you want to try many different learners and limit each of them to a single thread

Improvements to F# API and samples for ML.NET

Don Syme has been pioneering the work on driving improvements to the overall F# story for ML.NET. As Isaac’s issue had pointed out ML.NET did not support F# records. Work here is still ongoing but with 0.4 release ML.NET allows use of property-based row classes in F#. You can learn more about Don’s work as a part of this PR.

As a part of this change we have also updated the dot.net machine learning samples repo to add the language pivot for ‘fsharp’ porting over the existing samples to work for F# as well. We would love for you try them out and contribute more!

Help shape ML.NET for your needs

If you haven’t already, try out ML.NET you can get started here.  We look forward to your feedback and welcome you to file issues with any suggestions or enhancements in the GitHub repo.

https://github.com/dotnet/machinelearning

This blog was authored by Cesar de la Torre, Gal Oshri and Ankit Asthana

Thanks,
ML.NET Team

Today, we’re pleased to introduce the new and updated Developer’s Guide to Microsoft Azure eBook. Featuring extensive updates since the last update, the new eBook is designed to help you get up to speed with Azure in the shortest time possible and includes practical real-world scenarios.

This book includes all the updates from Microsoft Build, along with new services and features announced since then. In addition to these important services, we wanted to focus on practical examples that you’ll use in the real world and included a table and reference architecture that show you “What to use When” for databases, containers, serverless scenarios and more. We also put a key focus on security to help you stop potential threats to your business before they happen. You’ll also see brand new sections on IoT, DevOps and AI/ML that you can take advantage of today.

In the 20+ pages of demos, we’ll be diving into topics such as creating and deploying .NET Core web apps and SQL Server to Azure from scratch, to extending the application to perform analysis of the data with Cognitive Services. After we have our app we’ll make it more robust and easier to update by incorporating CI/CD and more. We’ll also see just how easy it is to use API Management to control our APIs and generate documentation automatically.

The eBook (PDF available for download) covers the following chapters:

• Chapter 1: The Developer’s Guide to Azure
• Chapter 2: Getting started with Azure
• Chapter 3: Securing your application
• Chapter 4: Adding intelligence to your application
• Chapter 5: Working with and understanding IoT
• Chapter 6: Where and how to deploy your Azure services
• Chapter 7: Microsoft Azure in Action
• Chapter 8: Summary and where to go next

We’re also pleased to announce that we will have EPUB and Mobi (“Save link as” or on Mac: Option+Click / Windows: Alt+Left Click) support in addition to PDF. You can also download all formats at once if you wish. Now you have multiple options and can get up to speed with Azure using your eReader or tablet of choice.

What are you waiting for? Download Now and sign up to be notified of future updates to the guide to ensure you make the most of the platform’s constantly evolving services and features.

Thanks for reading and keep in mind that you can learn more about Azure by following our official blog and Twitter account. You can also reach the author of this post on Twitter.

We are excited to announce the general availability of Azure SQL Database reserved capacity, now available for single and elastic pool databases—expanding our commitment to making Azure the most cost-effective cloud for your workloads. This new pricing option enables you to save up to 33 percent¹ compared to license-included pricing by pre-paying for your SQL database vCores for a one or three-year term.

Azure is the most cost-effective cloud for SQL Server workloads

Today, customers with active Software Assurance can save up to 55 percent using Azure Hybrid Benefit for SQL Server with the new vCore-based purchasing model in SQL Database. With support for reserved capacity on single databases and elastic pools, you can unlock even more savings when you combine your Azure Hybrid Benefit with reserved capacity pricing to achieve savings of up to 80 percent². 

If you’re facing end of support for SQL Server 2008/2008 R2 on July 9, 2019, you can migrate your workloads to Azure SQL Database, a fully-managed service, and eliminate end of support transition costs. Azure Hybrid Benefit lets you use your existing licenses with active Software Assurance to save more on Azure, and now reserved capacity makes the move to Azure SQL Database even more cost-effective.

Apply it to a single subscription or share across your enrollment

SQL Database reserved capacity pricing is scoped to either a single subscription or shared. It allows you to control how many subscriptions can use the benefit and how the reservation is applied to specific subscriptions.

A reservation scoped to a single subscription means that the reserved capacity discount will apply to SQL Database resources within the selected subscription. A reservation with a shared scope means that the billing benefit can be shared across any subscription in the enrollment (i.e., Enterprise Agreement customers) or account (i.e., Pay-as-you-go customers), except for MSDN subscriptions in the enrollment or non-Pay-as-you-go subscriptions in the account.

Keep your savings when you need to make changes

SQL Database reserved capacity provides vCore size flexibility so that you can scale up or down within a performance tier and region without impacting your reserved capacity pricing. For example, if you purchase 16 Gen5 vCores of reserved capacity for SQL Database general purpose and decide to deploy a 4 vCore database and an 8 vCore elastic pool, both with Gen 5 compute, your reserved capacity benefit will automatically apply to both. If you decide to scale up your database to 8 vCores, the reserved capacity benefit will also automatically apply.

You also have the flexibility to temporarily move your hot databases between pools and single databases as part of your normal operations, within the same region and performance tier, without losing the reserved capacity benefit. By keeping an un-applied buffer in your reservation, you can effectively manage the performance spikes without exceeding your budget.

SQL Database reserved capacity is initially available to single databases and elastic pools, with managed instance support coming in the future, and can be purchased through the Azure portal. For more information on reserved capacity pricing, please visit the SQL Database pricing page and review our documentation.

¹ Savings based on eight vCore SQL Database managed instance general purpose in West2 US region, running 730 hours per month. Savings are calculated from on demand full price (license included) against 3-year reserved capacity license Included. Actual savings may vary based on region, instance size, and performance tier. Prices as of May 2018, subject to change.

² Savings based on eight vCore SQL Database managed instance business critical in West2 US region, running 730 hours per month. Savings are calculated from on demand full price (license included) against base rate with Azure Hybrid Benefit plus 3-year reserved capacity. Savings excludes Software Assurance cost for SQL Server Enterprise edition, which may vary based on EA agreement. Actual savings may vary based on region, instance size, and performance tier. Prices as of May 2018, subject to change.

Interested in deploying your Linux or containerized web app in an Azure Virtual Network? The Azure App Service team is excited to announce the general availability of Linux on Azure App Service Environment (ASE), which combines the features from App Service on Linux and App Service Environment. As announced at the time of the public preview release, Linux customers will be able to take advantage of deploying Linux and containerized apps in an App Service Environment, which is ideal for deploying applications into a VNet for secure network access or apps running at a high scale. 

What can I do with Linux on ASE?

With Linux on ASE, you can deploy your Linux web applications into an Azure virtual network (VNet) by bringing your own custom container, or just bring your code by using one of our built-in images.

• If you want to bring your own custom Docker container, you can bring your image from DockerHub, Azure Container Registry, or your own private registry.
• If you want to use one of our built-in images, we support many popular stacks, such as Node, PHP, Java, .NET Core, and more to come.

Windows, Linux, and containerized web applications can be deployed into the same ASE, sharing the same VNet. Remember that even though Windows and Linux web apps can be in the same App Service Environment, Windows and Linux web apps must be in separate App Service plans. With Linux on ASE, you will be using the Isolated SKU with Dv2 VMs and additional scaling capabilities (up to 100 total App Service plan instances, between Windows and Linux, in one ASE).

To decide which kind of ASE is the best for your use case, you should think about what kind of IP you want to use to expose the apps hosted in your ASE. Will it be public or private? Depending on whether or not you want an Internet accessible endpoint, there are two different kinds of ASEs that you can create:

• An external ASE with an Internet accessible endpoint.
• An internal ASE with a private IP address in the VNet with an internal load balancer (ILB).

Our documentation page provide steps to get started, and more context about how to configure networking for your ASE.

Pricing changes from preview

Effective July 30, 2018, Linux and containerized apps deployed in an App Service Environment have returned to regular App Service on Linux and App Service Environment pricing. The 50 percent discount on the Linux App Service Plan from the public preview has been removed for general availability and is no longer being offered.

New regions/countries added

Since public preview, we have now expanded Linux on ASE to all App Service on Linux’s 20+ regions/countries:

How to get started

You can create a Linux Web App into a new ASE by simply creating a new Web App and selecting Linux as the OS (built-in image), selecting Docker (custom container), or creating a new Web App for Containers (custom container).

If you need more detailed instructions, get started with creating your first Linux/containerized Web App into an ASE by following these instructions.

We’d love to hear what you think! Please leave your feedback on Linux on ASE using the Wed Apps forum.

Check out this blog post for answers to commonly asked questions about Linux on App Service Environment general availability!