The excitement at Microsoft Build 2018 kicked off on May 6 with the annual presentation of the Windows Developer Awards, which acknowledge the hard work that goes into making great applications.

In a room full of the some of the best and brightest Windows developers, recognition went to Application Creator of the Year, Game Creator of the Year, Reality Mixer of the Year, Design Innovator of the Year, and Ninja Cat of the Year.

Apart from the Ninja Cat of the Year award, which was selected by an internal team of Windows experts, the top applications were voted on by the developer community.

Application Creator of the Year: Leveraging the latest Windows 10 capabilities

Winner: Affinity Designer 

Affinity Designer is the fastest, smoothest, and most precise vector graphic design software available. Whether for work or fun, the application revolutionizes the creation of everything from marketing materials and websites, to icons and UI. Features and functionality include real-time performance, perfect color and output, and multiple disciplines.

Game Creator of the Year: Outstanding game contribution to the Microsoft Store

Winner: Luna

Using Windows Mixed Reality, Luna immerses you in “Bird’s” peaceful summer, interrupted by it swallowing the last piece of the waning moon and getting blown far from home. Players unscramble celestial puzzles and create miniature musical worlds. The aim is to unlock each level‘s tree, plant, and animal spirits to help Bird reunite the fragmented moon and find its way back home.

Reality Mixer of the Year: Creator demonstrating a unique mixed reality experience

Winner: Space Pirate Trainer

Space Pirate Trainer transports users to the 80s arcade cabinet games of yesteryear, using today’s immersive experience. Through Windows Mixed Reality, users can fight off relentless waves of droids, with all the weapons and gadgets a Space Pirate could ever need.

Design Innovator of the Year: Beautiful look and feel

Winner: Huetro for Hue

Huetro for Hue easily connects with the Hue lighting system and syncs across Windows 10 devices to create new experiences. The colorpicker enables users to select specific colors or create new scenes using favorite memories. Ambiences allow for dynamic light shows. Alarms, Cortana, or NFC can be setup for home automation.

Commercial Developer of the Year: Focused on an enterprise audience

Winner: Wrike

Wrike mission “is to make teams insanely productive.” A SaaS work management and collaboration platform, Wrike supports millions of users, in more than 15,000 enterprises in 120 countries to manage work streams and organize everything needed to complete projects in one spot.

Ninja Cat of the Year: Special recognition

Winner: Oren Novotny

Oren Novotny is the chief architect of developer operations and modern software at application maker Blue Metal. He was selected as Ninja Cat of the Year for his contributions to the Windows community and efforts to make life easier for other developers. He serves on the .Net Foundation Advisory Council, is a member of the Visual Studio ALM Rangers, and has been a Microsoft MVP for the last four years.

Congratulations to all the winners!

Look for full profiles of each of them on our Medium channel in the coming weeks.

The post Windows Developer Awards: And the 2018 winners are… appeared first on Windows Developer Blog.

This week at the BUILD conference, Microsoft announced that Power BI custom visuals will soon be available as charts with Excel. You'll be able to choose a range of data within an Excel workbook, and pass those data to one of the built-in Power BI custom visuals, or one you've created yourself using the API.

Since you can create Power BI custom visuals using R, that means you'll be able to design a custom R-based chart, and make it available to people using Excel — even if they don't know how to use R themselves. There also many pre-defined custom visuals available, including some familiar R charts like decision trees, calendar heatmaps, and hexbin scatterplots.

For more details on how you'll be able to use custom R visuals in Excel, check out the blog post linked below.

PowerBI Blog: Excel announces new data visualization capabilities with Power BI custom visuals

Today’s web developers face more challenges than ever before. New frameworks spring up left and right. The number of devices capable of running the web multiply by the day. And we strive as creators to enable an inclusive and trustworthy world where fundamentals like accessibility, privacy, performance, and security are key to success.

These challenges are an opportunity for builders and consumers of devtools, requiring our solutions to enable innovation and scale. We need to provide web developers and tool vendors targeting Microsoft Edge, including other Microsoft teams, flexibility in solving these tough problems.

With the Windows 10 April 2018 Update, we’re taking the first major step on our journey by introducing the Microsoft Edge DevTools Protocol, a set of REST and JSON-RPC/WebSocket APIs, which enable clients to diagnose and debug Microsoft Edge tabs. In addition, we’ve worked with our teammates from Visual Studio, Visual Studio Code, and the Microsoft Edge DevTools to integrate with, and validate the interoperability of, these new capabilities within those apps.

In this post, we’ll elaborate on why we’ve taken this approach and the exciting progress we’ve made so far. We’ll provide resources throughout to help you get started using these new technologies. At the end, we’ll look at our next steps and share the many ways you can provide feedback to help us shape the future of the Microsoft Edge DevTools Protocol.

Get started with the new local and remote debugging scenarios with the new documentation from aka.ms/edp-docs. Requries the Windows 10 April 2018 Update or the Microsoft Edge DevTools Preview app.

Rebooting the Microsoft Edge DevTools platform

A year ago, we set out to re-architect the Microsoft Edge DevTools platform with two general customer scenarios in mind: growing the number of devtools that work with Microsoft Edge, and expanding support for debugging Microsoft Edge on new device form factors.

Expanding opportunities in local and remote scenarios is a key motivator for the new architecture. More details on the components in the diagram above are provided below.

To tackle the realities described above and achieve both local and remote support, we came up with a short but impactful list of guiding principles:

• Decouple the platform and client components and release the client separately to allow for faster UX improvements and innovations
• Build the platform interfaces on standard technologies to foster a larger ecosystem of devtools with support for Microsoft Edge
• Build the platform in a way which allows devtools running locally to target instances of Microsoft Edge on remote Windows devices
• Align with other browser vendors to support ecosystem portability and efficiency

This work will span multiple releases, but we’ve already made some exciting progress. Let’s dive in.

Decoupling the Microsoft Edge DevTools platform and client

To enable us to deliver customer value more rapidly, we first looked to take the Microsoft Edge DevTools app and ship it separately from the Windows OS as a Store application.

But to ship via the Microsoft Store, we needed to build only on APIs which are publicly available to others. This lead to the creation of the new Edge DevTools Protocol (EDP). The protocol is comprised of two sets of APIs: REST and JSON-RPC via WebSocket. Through EDP, devtools can build experiences around invoking methods and subscribing to diagnostics and debugger events.

The REST APIs are primarily used to discover information about EDP such as the version, available targets, and the supported API surface. All response values are formatted as JSON objects.

The WebSocket APIs are used to issue debugger and diagnostics commands and subscribe to correlated events. The APIs are broken down into Domains, which further break down into Methods, Events, and Types. A simple example would be the Debugger.setBreakpointByUrl API. A client would issue a JSON-RPC command looking something like this to set a breakpoint based on a JS file URL and location:

By switching to this publicly documented APIs, we’ve begun the process of shipping a new Microsoft Edge DevTools Preview app which runs in part on EDP. You can read more about the new DevTools Preview app here.

Fostering an ecosystem of Microsoft Edge DevTools

By decoupling the client from the platform, we’ve started the work to achieve our second goal of enabling an ecosystem of devtools which support Microsoft Edge. HTTP and WebSocket are ubiquitous technologies in native and web development environments. This means clients written in many programming languages can integrate. We hope this vastly reduces the barriers to entry for those who want to build devtools for Microsoft Edge.

In addition to calling APIs, from within those tools developers will need a way to launch the Microsoft Edge DevTools Server. We’re excited to announce that in the April 2018 update, we’ve introduced a new command line parameter and Edge UX for doing that. Check out the capture below for a preview of that experience:

The animated gif above shows the new Microsoft Edge DevTools server command-line launching functionality added in the Windows 10 April 2018 update.

These command-line parameters for launching the Microsoft Edge DevTools Server, along with available HTTP and WebSocket APIs, can be found in the Microsoft Edge DevTools Protocol documentation. Combined, these capabilities enable developers to more easily build devtools which target Edge.

To prove the viability of this approach, we on the Web Platform team partnered with the Visual Studio and Visual Studio Code teams to support F5 debugging of JavaScript running in a Microsoft Edge tab. We look forward to sharing more on that front soon!

Supporting remote debugging

In addition to wanting to foster a community of devtools which target Microsoft Edge, we want to enable those tools to target Microsoft Edge running on a remote Windows device. Though this specific scenario is not yet supported, it shows our long-term thinking:

Imagine you’re a HoloLens developer working with WebVR. Trying to debug your web app in Microsoft Edge on the HoloLens would likely be a time-consuming activity. Without a mouse and keyboard, navigating of devtools would be a cumbersome experience. Much better would be to run devtools on your local dev machine and remotely target Microsoft Edge instances running directly on the HoloLens.

To accomplish that, we built a plugin for the Windows Device Portal (WDP) which hosts the same Microsoft Edge DevTools Server that runs locally when launched via the command-line utility explained above. When you enable WDP in the Windows For developer settings, this will automatically install and activate the EDP plugin.

With your device in Developer mode and the Windows Device Portal turned on, the EDP plugin is automatically installed and HTTP and WebSocket servers started.

Using the Remote tab in the new Microsoft Edge DevTools preview app, you can connect to a remote device, view its targets, and connect to and debug them. You can read more about that here and here.

Note: At this point we support remote devices (and virtual machines) running the Windows 10 April 2018 Update or higher on PCs.

Aligning with the devtools protocol community

When we set out to build the devtools protocol for Microsoft Edge, we had a choice: build something which aligns with industry precedents or create our own approach tailored for our browser. The choice was obvious to us so we opted to align with the Chrome DevTools Protocol (CDP).

In the early winter of 2017, we met with fellow web experts from Google, Mozilla, Apple, VS Code, and others to discuss how better to align our work around devtools protocols. The result of that discussion is the DevTools Protocol Web Platform Incubator Community Group (WICG). Through this new group, we hope to publicly explore what alignment looks like and how best we as devtools protocol vendors can benefit the community.

Participants from Google, Mozilla, Microsoft, and VS Code drove the first draft of the WICG goals.

Separate from the WICG, we’ve worked hard as we’ve implemented EDP to validate that our API surface and method/event semantics mirror CDP. By releasing version 0.1 with support for three clients (Visual Studio Code, Visual Studio, and Edge DevTools preview app), we’ve achieved a high degree of interoperability. Where needed due to platform differences, we’ve introduced Microsoft Edge-specific methods and properties, all prefixed with ms and safely ignorable by any client not requiring them.

Lastly, though we’ve committed to testable interoperability, we know there will be bugs, and we hope you will help us find them! We look forward to your feedback as you build great devtools using EDP.

Next steps

This release is the first major step on a multi-milestone journey. We’re passionately committed to building the best Microsoft Edge developer tools and platform APIs to enable our customers to be more efficient and to fix bugs in Microsoft Edge. In terms of what’s next for EDP, here’s a quick look:

• More devtools: we have more features to migrate to EDP within the Microsoft Edge DevTools, and are actively migrating the most-used tools to the new APIs. In parallel, we’re investigating how tools like Sonarwhal and WebDriver can integrate, as well.
• More devices: the first release supports Windows Desktop SKU devices (including VMs of that flavor), and we plan to expand the Windows SKUs and devices supported for remote debugging.
• Interop improvements: as more devtools on-board and we grow our API surface, we’ll continue to invest in ensuring we’re working with the larger DevTools Protocol community to align our efforts.

That’s a rough and quick run through our thoughts about the future. We’ll continue to evolve these priorities as we interact with our developer communities to build new platform capabilities together.

Providing feedback

If you’d like to provide feedback, we’d love to hear it and recommend several channels:

• Feedback Hub: You can file feedback using the Microsoft Edge category and the Developer tools sub-category in the Feedback Hub app on Windows 10—just press Win+F to get started. Whether it’s a bug or suggestion, this is the easiest way for us to receive, track, and communicate around issues you bring to our attention.
• Edge DevTools UserVoice: We monitor and use requests as an important signal informing what we do, and to help prioritize the order in which we implement new features. Please add requests or upvote your favorites.

Thanks for taking the time to learn about the new Microsoft Edge DevTools Protocol and our vision for a rich ecosystem of Microsoft Edge devtools. We hope you’re as excited as we are. Happy coding!

– Brendyn Alexander, Senior Program Manager

The post Introducing the Microsoft Edge DevTools Protocol appeared first on Microsoft Edge Dev Blog.

It's conference season, so I've been travelling on a lot of planes recently. (And today, I'm heading to Budapest for eRum 2018.) So this resonates with me right now:

There's another video in the same vein about hotels which is quite funny as well. Anyway, that's all from us for this week; we'll be back next week reporting from eRum. Until then, have a great weekend!

It has been about an year since the last Microsoft Build conference where we announced the Microsoft ad monetization platform – a server-side ad mediation platform from Microsoft that helps app publishers & web sites to maximize their ad revenue globally through innovative ad experiences & one stop shop integration, analytics & payout experiences in Windows Dev Center. It’s amazing to reflect on the progress we’ve already made in partnership with the developer ecosystem & Microsoft 1^st Party publishers like Xbox, Xbox Casual Games and Mixer. We are happy to share several new and exciting announcements for the developer ecosystem to accelerate their monetization opportunities.

Windows developer ad monetization progress report

One of the key promises of the Ad monetization platform has been to help improve ad yield i.e. revenue per impression. Better Ad yield is a function of advertiser ROI i.e. advertisers bid more if they see higher value for the impression. One of the ways to improve the yield is through diverse set of advertisers participating in the bidding process. We’re happy to report that we have added many high-quality ad partners (typically these are advertising platforms where many advertisers bid on inventory) such as Taboola, Outbrain and Revcontent to serve demand on our ad inventory and have more ad partners across key markets participating in the upcoming months. Thanks in part to this, we are pleased that the total developer ad revenue for the July-December period has grown by 35% YoY. Ad yield for Windows developers on the UWP platform has more than doubled over the last year due to addition of strategic ad partners as well as our programmatic evangelism efforts like creation of the Windows Premium Ads Publisher program etc.

Figure 1: Ad partners that currently serve across ad formats for UWP developers

We are also very excited to announce the addition of a new ad partner, called “MSN Content Recommendations” to deliver Ads on banner and Native placements. Users will get to see Ads with rich content from MSN across several different verticals such as finance, health and fitness, lifestyle etc. The addition of this Ad partner will further help improve the fill rates for banner ad content toward 100%.

Competition between Ad partners augurs well for developers as there are more contenders for delivering quality ads, automatically improving yield for developers in the marketplace. We have made noteworthy progress with our machine learning algorithms which optimize every ad request in real time to maximize revenue for the developer. For every ad unit that is set to “Auto” optimization, we send each ad request to the most optimal configuration of ad networks chosen based on the app in context, user and device signals and past performance of each ad network on that ad unit. We’re happy to note that our improved algorithms have generated a 20% improvement in Ad click through rates and 15% improvement in RPM (revenue per thousand impressions) compared to the prior model.

Interstitial banner is an ad format that highlights the advertiser’s message more efficiently and has a higher engagement from users, leading to greater clicks and higher revenue. With the introduction of interstitial banner support in July 2017, we have seen high eCPMs for developers who have adopted this ad format into their apps. We added support for Revcontent since then and will be adding few more partners such as ‘MSN Content Recommendations’ and Undertone in the coming weeks.

Native ads:

We introduced support for native ads in the Microsoft Advertising SDK as an invite only preview program for game publishers & other app developers during /build 2017. During this phase, we added several native ads partners such as AppNexus, Microsoft app install ads and Revcontent to serve on the native ad placements. We are working actively to bring additional demand partners such as MSN Content Recommendations and Taboola in the coming weeks. These efforts will pay off for improving the yield for developers that build modern applications monetized through native ads.

Today, we are happy to announce that we are exiting the pilot phase and the Native Ads experiences are generally available to all developers.

One of our leading developers, PicsArt, who had been participating in the Native Ads pilot, had this to say about their Native Ads development experience:

“Implementation of Native ads was a smooth and seamless experience. From creation of ad unit to it’s addition in the application was easy and made clear with the help of the comprehensive documentation.”

— PicsArt

Figure 2: A Native Ad from the PicsArt Photo Studio application

Universal user acquisition & engagement platform enhancements

We are happy to announce new set of capabilities to enhance the user acquisition and engagement capabilities in the Windows Dev center.

1) Acquire Users outside Microsoft Network: All the ad campaigns that you created up until now used to serve on the Microsoft’s 1^st and 3^rd party app and web properties as depicted in the picture below. However, some developers may have specific requirements for high volume of acquisitions or may be looking for very niche audience segments. Starting today, such ad campaigns by default will start serving on external websites and apps running on Windows 10 in addition to Microsoft internal ad surfaces. This feature will help developers find millions of users who can potentially download/use their app outside Microsoft’s 1^st and 3^rd party ad surfaces.

Figure 3: Difference between existing and new ad display surfaces

2) Video App Install Ads: We’re also announcing the beta program for video app install ads product where developers can use video ad creatives along with the banner/interstitial formats that we already support. We have seen amazing results with the top developers while running tests with CTR’s of around 5% when using video creatives which is almost 20X of the banner creatives. Though in beta, the program is open to all with campaign budgets higher than $10,000 and pricing of the video creatives for the first three months starting today will be $0.40 per thousand impressions. Post the promotional period, video rates will match with the industry averages for respective markets and will be auto-tuned by the ML algorithms to get the best cost per install during ad auctions.

3) Notification Analytics: Targeted push notifications are getting enhanced reports where the only available metrics were related to delivery rate and app opens until now. We will now give detailed reports for targeted notifications including positive clicks in the form of click through rate [CTR] and counts of actual actions by users on the notifications like click/dismissal/snooze/decay etc. Below are a few sample reports of the latest experiences that you will see in the notifications page starting today.

Figure 4: New notification metrics in the Windows Dev Center

Ad Viewability reporting in Dev Center

Advertising industry is moving towards valuing viewable impressions rather than just impressions delivered. Advertisers tend to bid higher for viewable impressions as they have an increased chance of their ads being seen by users.

We are happy to announce that you will now be able to measure the viewability of each of your ad units in the Dev Center Ads Performance Report.  The new ‘Viewability’ metric indicates the number of viewable impressions for every 100 ad impressions delivered. This metric allows you to determine the quality of your ad slot and allows you as a developer to make informed decision on ad placement in your app.

Figure 5: New Viewability Reports on DevCenter

For more information on the rules used to classify an impression as viewable and tips to increase viewability on your ad unit, please refer to MSDN.

Microsoft Ad Mediation platform for Android

We are excited to announce the preview of Microsoft Ad mediation platform for Android developers. This will bring in all the goodness of the Microsoft Ad mediation Platform to the benefit of existing and new Android developers. We have many exciting capabilities including iOS & Web Ads SDK planned later this year. Learn more about current features through our documentation and code samples.

Microsoft Ad mediation platform for Android developers brings the same mediation and optimization capabilities that are available to UWP app developers today. Also available are all the same premium Microsoft and third-party ad partners that are currently serving on UWP publisher surfaces.

This platform is open for all Android app developers and publishers and not just for developers registered with Windows DevCenter. Please reach out to us if you are interested in this platform and augment your Android apps with additional Ad revenue today.

Also read:

Below is a list of capabilities that we released over the past two years which can potentially help in your journey of ad monetization and user acquisitions. Do give them a read!

The post Announcing the General Availability of Native Ads & Preview of Microsoft Ad mediation for Android developers appeared first on Windows Developer Blog.

On a weekly basis, more than 5 million people find help using Microsoft online help and documentation. Despite having more than 380,000 pages of articles, references, and tutorials, we each have our own stories of painful gaps.

Today, we’re introducing a new project to help, code-named Aladdin.

What is Aladdin?

After installing the Aladdin extension, “Related Questions” prompts will appear on Microsoft Docs article pages:

Each prompt recommends contextually-relevant articles:

 
Click on a card for more information, and “go to original article'” to navigate to the source page in a new tab:

Ask Aladdin if you have questions – Aladdin surfaces expert responses to questions from real customers:

Install Aladdin

Install the Aladdin extension for your Chrome browser. Aladdin will now be there to assist you on Azure documentation pages. Edge support is coming soon!

How does it work?

Aladdin builds a knowledge base from Azure documentation, customer comments, and visitor telemetry. Using this knowledge base, along with Azure Cognitive Services, Aladdin connects you to relevant resources and helps you get your job done faster.

What data do you collect?

The Bio-IT World Conference and Expo kicks off tomorrow in Boston. Microsoft is a platinum sponsor of this event. The research and product development being done in the life sciences and healthcare industries are having a tremendous impact. Many of these efforts require a large volume of computation. Bio-IT World is where thought leaders at the forefront of this work come together to share knowledge and help advance the state of the science.

Representatives from our genomics team will be on hand in booth #446 to showcase Microsoft’s solutions for secure and scalable genomics work. Our Azure Specialized Compute team will demo how Azure’s high performance computing offerings can help researchers get simple, managed access to the compute resources they need, whatever that need may be. In addition, our partners at DNAnexus will be presenting a demo in the Microsoft booth from 3:30 to 4:00pm on Wednesday.

Jason Stowe, Principal Group Program Manager in the Azure Specialized Compute team will be an instructor for the Bootstrapping your life science startup workshop on Tuesday afternoon. On Wednesday morning, he will introduce the plenary keynote. In this session, a panel of experts will discuss data science issues in life sciences.

Jason will also give a talk titled Solving practical challenges for futuristic workloads in the cloud track at noon on Wednesday. This talk shares lessons from over a decade of helping customers adopt cloud for traditional HPC workloads as well as new *omics and AI workloads. Jason presents lessons from cloud adopters including how to achieve the governance you need to accelerate life sciences innovation.

If you’re at Bio-IT World, we look forward to seeing you in booth #446.

Gaining insights rapidly from data is critical to competitiveness in today’s business world. Azure SQL Data Warehouse (SQL DW), Microsoft’s fully managed analytics platform, leverages Massively Parallel Processing (MPP) to run complex interactive SQL queries at every level of scale.

An enterprise data warehouse doesn’t exist in isolation but is part of a complete analytics solution that includes the ‘pipelines’ that extract, transform, and load data, often from many disparate sources, as well as staging and reporting layers. Tasks such as continuous integration and continuous delivery require extensive manual coding and doesn't offer the flexibility needed for the rapidly changing business needs. This is where data warehouse automation tools provide value for customers.

Data warehouse automation (DWA) tools are meta-data driven, code generation tools that streamline developing and managing a data warehouse solution. DWA tools provide more than just ETL automation, they automate the complete life cycle of a data warehouse solution, from analysis, design, and implementation to documentation, monitoring and maintenance.

Benefits of data warehouse automation

Data warehouse automation (DWA) tools bring the benefits of meta-data driven automation, and code generation to streamline developing and managing a data warehouse solution. DWA tools provide more than just ETL automation, they automate the complete life cycle of a data warehouse solution, from analysis, design, and implementation to documentation, monitoring and maintenance.

Acceleration: By automating repeatable, tedious tasks such as source discovery, fact and dimension mapping, dimension type assignment, change-impact analysis and version control, DWA tools accelerate time to market for data warehouse projects.

“WhereScape automates the design, development and operation of agile data warehouse projects by increasing developer productivity by 10x. It leverages the enormous processing power of Azure SQL DW’s MPP architecture together with the ease of use and support that comes with using T-SQL as the processing and query language.”

– Douglas Barrett, Senior Solutions Architect, WhereScape

Consistency: By leveraging data warehouse design patterns and platform specific best practices, these tools deliver consistent, high quality code. Automatically-generated documentation improves understanding and helps with troubleshooting.

“It’s exciting to see BimlFlex customers benefiting from our Azure SQL DW automation experience. We provide optimal implementations that employ patterns approved by industry experts and Microsoft product teams - and can be selected or changed through simple modifications to our metadata store.”

– Scott Currie, CEO, Varigence

Simplicity: Graphical user interface (GUI) with point-and-click design capability offered by most DWA tools, simplify the user experience for source exploration, modelling fact and dimension tables, designing ETL pipelines and reporting.

Flexibility: By shortening iteration cycles, DWA tools help IT and business teams focus on business requirements and reduce the time-to-insight, increasing the business value of data warehousing.

“AnalytiX DS Epicenter’s metadata driven automation platform allows code to be automatically generated to truly accelerate time to value for customers while automatically documenting the data integration processes in Epicenter’s enterprise data mapping platform”

– Michael Boggs, CTO and Founder of AnalytiX DS

SQL DW and data warehouse automation partners

The following data warehouse automation partners have certified their tools with SQL DW:

“We help data driven companies be more successful with their data management. Azure SQL DW along with our powerful automation approach for designing, generating and deploying analytical solutions enables our customer to get maximum value out of their data. biGENiUS delivers optimized generator leveraging massively parallel processing capabilities of Azure SQL DW.”

– Gregor Zeiler, Senior Solution Manager, biGENiUS

The following partners have support for SQL DW coming soon:

“We are excited to be providing support for Azure SQL DW in our efforts to align our road-map with our customer’s choice of cloud data warehouse. Dimodelo Architect will speed up the transition, migrate existing data warehouse data, and automate development processes to realize the value from their cloud investment faster.”

– Adam Gilmore, Chief Product Officer, Dimodelo Solutions

“Discovery Hub by TimeXtender is a modern software platform that allows customers to build, deploy and manage a complete enterprise-grade, analytical architecture. We are pleased to be adding support for Azure DW, to complement our already broad Azure Data Services offering, in our Fall 2018 Discovery Hub release.”

– Heine Krog Iverson, CEO TimeXtender

Customer testimonials

Read what customers have to say about data warehouse automation tools:

“We started our cloud journey last year with SQL Data Warehouse to leverage its MPP architecture and the ease of scaling. It is fast becoming the backbone of all our corporate insights. To accelerate the development and to provide consistency across loading packages, we decided to use Varigence's BimlFlex as it ticked all boxes for our acceptance criteria for loading patterns, technologies and prices. We have currently been using BimlFlex to load our production Azure SQL DW for over six months.”

– Stephen Townsend, Business Intelligence Architect, an energy company

“With data warehouse automation, I define structural patterns that I consistently reuse - almost like a library I can import. It allows my team at Ashley to spend less time focusing on building out specific ETL and DW processes in the code and more time collaborating with end user on their business needs.”

– Phil Haddad, Data Architect, Ashley Furniture

“At Smurfit Kappa, we have been using biGENiUS to improve and maintain our reporting system that we have developed significantly over the last ten years. Reduction in development time, simplified loading experience, ease of disparate source addition and traceability are some of the key benefits that we gained from using a data warehouse automation tool.”

– Christian Foucher, Manager BI and Shared Applications, Smurfit Kappa

“With AnalytiX DS Enterprise Mapping and Automation platform we were able to reduce overall project duration by close to 25 percent and cost by close to 35 percent. Our deliverable’s have also improved considerably due to fact all development resources now have a common interface to a centralized metadata repository which provides us with complete process, flow and version control capability.”

– Tony Jurek, Practice Principal, DXC Technology

Get started

Get started with data warehouse automation tools by checking out the free product trial or request a demo: AnalytiX DS, AnalyticsCreator, Varigence, WhereScape, and biGENiUS.

Learn more about SQL DW and stay up-to-date with the latest news by following us on Twitter @AzureSQLDW.

Since the release in 2016, developers are using our Azure IoT Python SDK to write device and back-end applications to connect to Azure IoT Hub and Device Provisioning Service, as well as writing modules for Azure IoT Edge (preview). Python is a popular choice for prototyping, and it is gaining traction in the embedded world.

If you decide to use the Python SDK for development, there are few things you should keep in mind: Python SDK is a wrapper on top of our Azure IoT C SDK, and we release binary packages on pip for Windows, Ubuntu, and Raspbian, all of which are compatible with Python 2 and Python 3. This approach has its ups and downs. On the upside, the features you see in C are available in Python with no functionality difference. On the downside, it is not a native Python SDK. Application Programming Interfaces (APIs) that we exposed may look different from typical Python APIs; and as a developer, you will need to pay attention the architecture of underlying platform, especially when you are using pip!

At a high level, there are three things that must align to reference the Python SDK properly: Python version (2 or 3), system architecture (x86 or x64) and pip reference (Python 2 or Python 3). This combination leads to common errors such as packages are installed using pip for Python 2, but the script is run with Python 3; Python x86 is installed for an x64 machine, and more. These tips should help with common errors based on your development environment.

Windows

Windows does not have Python pre-installed. The default installer on python.org is x86, which may not be the architecture of your PC. The processor type information is available in Windows Settings. If your system is x64 processor-based, you need to download the x86-64 installer from Python.

Ubuntu

Ubuntu has Python 2 installed and it cannot be removed from the OS. However, if you upgrade the system using sudo apt, be sure to install Python 3 on the machine. Ubuntu still uses Python 2 by default with pip, so there won’t be any issues if you are trying to use Python 2. If you plan on using Python 3, be sure to install the SDK using pip for Python 3 and run the script with Python 3.

Raspbian (Raspberry Pi)

Raspbian is a distribution of Linux, so it has similar behavior to Ubuntu. Python 2 is installed and cannot be removed. When you upgrade the system, Python 3 is also installed on the machine. Unfortunately, pip defaults to Python 3 but the OS defaults to Python 2 when you try to run the script. This causes a not found error because the package is installed on a different Python.Python 2 users should install with pip for Python 2, and Python 3 users should run with python3 instead of python.

MacOS

Currently, the Python SDK does not provide pip packages for MacOS. This is work in progress and will be available soon. In the meantime, you can build the SDK following instructions.

We hope this has been helpful! We are continuously working on improving the experience for Python developers, including working with the boost library and using pip packages. Stay tuned for more announcements in this area!

Build 2018

Last week, thousands of developers gathered in Seattle for Microsoft Build 2018, Microsoft’s annual developer conference, to learn how we are empowering them to advance the future of society with AI and the intelligent edge. This week's Azure.Source covers a lot of content. For a more concise (TL;DR) summary of Build 2018, check out the Microsoft press release, Microsoft Build highlights new opportunity for developers, at the edge and in the cloud. For a roll-up of all the Azure announcements, see: Azure Announcements from Microsoft Build. And if you want to dig into all of the content delivered at Build 2018 session-by-session, you can now do so at Microsoft Build Live 2018.

New Azure innovations are helping developers write code today for tomorrow’s technology challenges - Scott Guthrie's technology keynote at Build 2018 focused on the key areas of new Azure innovation that enable the intelligent cloud and intelligent edge – spanning developer tools, DevOps, containers, serverless, Internet of Things (IoT), and artificial intelligence (AI).

Azure AI Platform announcements: New innovations for developers - Joseph Sirosh, Corporate Vice President, Artificial Intelligence & Research provides an overview of the announcements made at Build for the Azure AI Platform, which consists of AI Services, AI Tools & Frameworks, and AI Infrastructure. In addition, Cognitive Services add pre-built, cloud-hosted APIs for developers to add AI capabilities, including new services announced at Build. This post also covers Cognitive Search and Azure Machine Learning (ML) advancements.

The Microsoft data platform continues momentum for cloud-scale innovation and migration - Rohan Kumar, Corporate Vice President, Azure Data covers announcements of recent Microsoft investments that continue to empower every developer, data scientist, and data professional to do more with data. Unleash the power of data with integrated and highly scalable data and analytics solutions to help organizations capture, store, and process, all data in real-time for driving intelligence into applications, to the edge, and back across the organization.

Azure Networking May 2018 announcements - Yousef Khalidi, Corporate Vice President, Azure Networking covers new networking services we have launched as well as enhancements we have made. Launching new services such as DDoS, VNet access to Azure services, zone-aware Application Gateways, a new global scale CDN offering, along with a new and super-fast Load Balancer, we continue to enhance the networking capabilities of Azure and more importantly develop new services and technologies to help customers run, manage, and achieve more when running their most demanding workloads.

Azure confidential computing - Mark Russinovich, CTO, Microsoft Azure shares more details about our confidential cloud vision and the work we’ve done since the announcement of our Azure confidential computing efforts. Azure Confidential Computing is aimed at protecting data while it’s processed in the cloud. Delivering on this vision requires us to innovate across hardware, software, and services that support confidential computing: hardware, compute, development, attestation, services/use cases, and research. Learn how you can sign up to request preview access.

AI

Microsoft empowers developers with new and updated Cognitive Services - Learn about all of the new updates that came to Cognitive Services at Build for vision, speech & machine translation, language, knowledge, search, and the new Cognitive Services Labs. Cognitive Services Labs provides an early look at technologies that are still in the experimental stage.

Microsoft Conversational AI tools enable developers to build, connect and manage intelligent bots - Natural interactions with computers start with language, speech, and semantic understanding, and continues through supporting rich multi model interactions. This blog post provides a brief recap of all Conversational AI announcements from Build and takes a quick dive into some of our newly updated services, such as the Bot Framework Emulator.

Announcing Cognitive Search: Azure Search + cognitive capabilities - Cognitive Search is an AI-first approach to content understanding. Cognitive Search is powered by Azure Search with built-in Cognitive Services. It pulls data from a variety of Azure data sources and applies a set of composable cognitive skills which extract knowledge. This knowledge is then organized and stored in a search index enabling new experiences for exploring the data.

Build 2018: What's new in Azure video processing and video AI - We announced a major new API version for Azure Media Services, along with new developer focused features, and major updates to Video Indexer. Developers can now begin working with the public preview of the new Azure Media Services API (v3). See Build 2018: Video Indexer updates for more info.

Full-integrated experience simplifying Language Understanding in conversational AI systems - LUIS brings together cutting-edge speech, machine translation, and text analytics on the most enterprise-ready platform for creation of conversational systems. Learn about advancements in text analytics, translation middleware for bots, improvements to understanding entities and intents, and how we're democratizing ML further.

Introducing Project Kinect for Azure - Building on the technology that debuted with Kinect and became a core part of HoloLens, Project Kinect for Azure combines Microsoft’s next-generation depth camera with the power of Azure services to enable new scenarios for developers working with ambient intelligence. This technology will transform AI on the edge with spatial, human, and object understanding, increasing efficiency and unlocking new possibilities.

IoT

What’s new with Azure IoT Edge? - Learn about the key innovations you will find when Azure IoT Edge is generally available in the next couple of months, including: AI on Azure IoT Edge, Azure IoT Edge core technology, Azure IoT Edge security, using Kubernetes to managed Azure IoT Edge deployments, the Azure IoT Edge modules in the Azure Marketplace, and Azure IoT Edge hardware certification.

Developing an Azure Sphere experience with Visual Studio - When looking at the landscape of MCU (Microcontroller Unit, not Marvel Cinematic Universe) development options, we knew we needed to deliver a better experience, which is why we developed Visual Studio Tools for Azure Sphere. The Visual Studio Extension for Azure Sphere adds new application templates to Visual Studio which you can use as the starting point for your application as well as SDK tooling for building and deploying your applications to Azure Sphere devices. Learn how you can pre-order an Azure Sphere development kit from Seeed Studio.

Announcing Azure Maps - Azure Maps (fka, Azure Location-Based Services) is now generally available and provides Microsoft enterprise customers with robust mapping capabilities to integrate into their various applications. Learn what else changed besides the name from this blog post, and from the Azure Friday series below.

Enabling more device management scenarios with new features in IoT Hub - Learn about two new features released in public preview at Build: automatic device management, and module identities and module twins. Learn about the scenarios these features enable.

Manage Azure IoT Edge deployments with Kubernetes - Learn about an experimental project which lets you leverage your Kubernetes expertise and use the same vocabulary to manage IoT Edge deployments right from within your Kubernetes environment.

Azure IoT solution accelerators bring the next wave of innovation - Learn about the next stage in our evolution of customizable solutions: from Azure IoT Suite to Azure IoT solution accelerators. Check out the Azure IoT Reference Architecture, a comprehensive guide for developers and architects wanting to understand technological choices behind a cloud solution for the Internet of Things, and the new Azure IoT Solutions site.

Azure Cosmos DB

Azure #CosmosDB @ Build 2018: The catalyst for next generation apps - We announced several new capabilities at Build 2018 - all of which are intended to enable you to easily build your mission-critical, globally distributed apps. Get a full overview of these announcements in this comprehensive post and read what customers have told us the transformative role Azure Cosmos DB has played in their businesses.

Introducing the #Azure #CosmosDB Bulk Executor library - Learn about the Bulk Executor library (available in .NET and Java), which enables you to perform bulk operations in Azure Cosmos DB through APIs for bulk import and update.

Virtual Network Service Endpoints for Azure #CosmosDB is now generally available - Azure Cosmos DB uses Virtual Network Service Endpoints to create network rules that allow traffic only from selected Virtual Network and subnets. This feature is now available in all regions of Azure public cloud. Read this post to learn how to use this feature by enabling service endpoint for Azure Cosmos DB for the subnet of a Virtual Network.

Sharing provisioned throughput across multiple containers in Azure #CosmosDB - In addition to provisioning throughput for an individual container in Azure Cosmos DB, you can now provision throughput collectively, for a set of containers as well, which allows those containers to share the provisioned throughput. This dramatically simplifies the capacity planning, since you can now configure provisioned throughput at different granularities based on your data model or application needs. Learn how this dramatically simplifies the capacity planning, since you can now configure provisioned throughput at different granularities based on your data model or application needs.

Have fun and draw awesome on the cosmic scale Azure #CosmosDB #PxDraw canvas today! - PxDraw is a massive, realtime drawing canvas, that allows you build pictures one pixel at a time in unison with thousands of other people from around the world. PxDraw is built with a variety of Azure and Microsoft technology including Azure Cosmos DB, Azure Functions, Azure Storage, SignalR, C#, TypeScript, and more. It's designed to showcase lots of writes across a lot of regions while maintaining low latency by utilizing Azure Cosmos DB's mutli-master capability which is now in limited preview.

Announcing new Async Java SDK for Azure #CosmosDB - Learn about a new asynchronous Java SDK for Azure Cosmos DB’s SQL API, which is open sourced on GitHub. This SDK leverages the popular RxJava library to add a new async API surface area for composing event-based programs with observable sequences.

Containers

Azure Container Registry makes geo-replication generally available, adding lifecycle management capabilities - Azure Container Registry geo-replication is now generally available. ACR Build provides inner-loop and trigger-based container build capabilities, and is now available in preview. Lastly, secure by default container registries are now in private preview. Read this post to learn more about these capabilities.

Monitoring Azure Kubernetes Service (AKS) with Azure Monitor container health (preview) - You can now rely on Azure Monitor to track the health and performance of your AKS cluster. Read this post to look at the new container health monitoring capability in Azure Monitor, which now is preview.

Microsoft syndicates container catalog (mcr.microsoft.com) - This month, Microsoft will start serving container images from mcr.microsoft.com, the Microsoft Container Registry (MCR), and syndicating the content describing the container images with Docker Hub and Red Hat. Learn how this syndication model will enable multiple channels of discovery, with a single download source for Microsoft’s container images.

Kubernetes on Azure: Industry’s best end-to-end Kubernetes experience - Read about the simple change of Azure Container Service becoming Azure Kubernetes Service, and then dig into announcements for making AKS easier than ever, new AKS features, and Dev Spaces for Kubernetes.

Now in preview

App Service: Adding multi-container capabilities and Linux support for App Service Environment - Announces public previews for multi-container support, Linux support for App Service Environment (ASE), and simplified diagnostics, debugging, authentication, and authorization.

Azure SignalR Service, a fully-managed service to add real-time functionality - Announces the public preview of the Azure SignalR Service, which is a fully-managed service that enables you to use ASP.NET Core SignalR to build real-time experiences such as chat, stock tickers, live dashboards, and more, all without worrying about capacity provisioning, scaling, or persistent connections.

Announcing Microsoft's own Content Delivery Network - Announces the public preview of Microsoft as a provider within Azure CDN, enabling Azure customers to use and deliver content from Microsoft’s own global CDN network, which is being added as an option alongside existing provider options from Verizon and Akamai.

Break through the serverless barriers with Azure Functions - Announces the general availability of durable functions, but also announces the preview of Functions Diagnostics by leveraging App Service Diagnostics.

Simplifying blockchain app development with Azure Blockchain Workbench - Announces the public preview release of Azure Blockchain Workbench, a new offering that can reduce application development time from months to days. Workbench gets customers started quickly by automating infrastructure setup, so developers can focus on application logic, and business owners can focus on defining and validating their use cases.

Introducing the Azure Terraform Resource Provider - Announces a private preview of a new Azure Resource Provider for HashiCorp Terraform that will enable Azure customers using Azure Resource Manager (ARM) to provision and configure dependent resources with Terraform Providers as if they were native Azure Resource Providers.

Better integrations and higher productivity with Azure Event Grid - Announces public preview support for a host of new features aimed at improving your developer experience, expanding the list of Azure services that natively integrate with Event Grid, and embracing standards that allow cross cloud interop.

New Azure Stack developer features to build intelligent, hybrid applications - Provides a rundown of announcements and a recap of a few key updates since launch. Also announces the preview releases of Templatized Service Fabric and Kubernetes Clusters, which will help you to run microservices and container-based workloads using the IaaS capabilities of Azure Stack.

Shared Image Gallery now in limited preview - Announces the limited preview of the Shared Image Gallery, which offers an easy but powerful set of tools to share VM images on Azure. Request your invitation by signing up to express interest.

Azure Service Fabric at Microsoft Build 2018 - Provides announcements and updates about Service Fabric, such as the private preview of Azure Service Fabric Mesh, which offers the same reliability, mission-critical performance and scale customers get with Service Fabric, but no more overhead of cluster management and patching operations.

4 new features now available in Azure Stream Analytics - Announces the public preview of Session windows, which group events that arrive at similar times, filtering out periods of time where there is no data. In addition, three private previews are now available for C# custom code support for Stream Analytics jobs on IoT Edge, Blob output partitioning by custom attribute, and updated Built-In ML models for Anomaly Detection.

Azure Event Hubs for Kafka Ecosystems in public preview - Announces the public preview of Event Hubs for Kafka Ecosystem, which provides a fully-managed cloud service alternative to installing and running your own Kafka clusters.

Now generally available

Azure Database Migration Service now generally available - Announces general availability of the Azure Database Migration Service (DMS), which is designed to enable seamless database migrations with minimal downtime from a variety of sources to the Azure data platform. Azure DMS is tuned to ensure the optimal migration experience, whether upgrading from legacy versions of Microsoft SQL Server or moving from sources such as Oracle, Sybase, DB2, MySQL, and others.

Azure mobile app is now generally available - Announces the general availability of the Azure mobile app, which brings the full power of Azure to your fingertips, no matter where you are. Azure mobile app is available for iOS and Android.

Also generally available

• Azure Cosmos DB Virtual Network Service Endpoints

News and updates

New Integrations for GitHub Developers - Announces the integration of GitHub with App Center and Outlook. After connecting a GitHub repo to App Center, developers can configure apps to build on commit, test automatically on thousands of real devices, distribute to beta testers, collect analytics and crash reports, and even upload to app stores when ready for release. Microsoft 365 supports Adaptive Cards, which streamline decision-making for users of Microsoft Teams and Outlook. GitHub has built an Adaptive Card into their notification emails to enable commenting on issues directly from your inbox, and soon you will be able merge pull requests, too.

Azure Marketplace is reaching new audiences - Azure Marketplace expands its offerings to include Consulting Services offers on App Source, subscription offers for SaaS ISVs, Private Offers on Azure Marketplace, and a new containers images category. Read this post for the details and dig into the Publisher Guide to learn how to participate.

Enhancements in Application Insights Profiler and Snapshot Debugger - Announces a series of improvements on Application Insights Profiler and Snapshot Debugger to ensure you can easily and conveniently use the tools.

Azure SQL Data Warehouse now supports automatic creation of statistics - Announces that Azure SQL Data Warehouse (Azure SQL DW) now supports automatic creation of column level statistics objects to provide greater flexibility, productivity, and ease of use for system administrators and developers, while ensuring the system continues to offer quality execution plans and best response times.

Extract management insights from SQL Data Warehouse with SQL Operations Studio - Use SQL Operations Studio with Azure SQL Data Warehouse (SQL DW) to create rich customizable dashboard widgets surfacing insights to your data warehouse and unlock key scenarios around managing and tuning your data warehouse to ensure it is optimized for consistent performance.

In case you missed it: 10 of your questions from our GDPR webinars - Get answers to ten of the most frequently asked questions arising in the recent GDPR webinar series, which are now also available for on-demand viewing.

Additional news and updates

• Deploy to Azure Kubernetes Service by using DevOps Projects: Sprint 134 Update
• June 2018 version updates to PHP

Azure Data Factory updates

• New connectors available
• Lookup activity supports a rich set of connectors
• GetMetadata activity supports more sources and metadata options

Azure Stream Analytics updates

• Blob output partitioning by custom attribute
• Updated built-in machine learning models for anomaly detection
• C# custom code support for jobs on IoT Edge
• Session windows

Power BI Embedded updates

• Custom report tooltips and Q&A explorer available
• Azure alerts available
• QPU High Utilization metric available
• Azure Memory Thrashing metric available
• Azure Memory metric available

Events

OpenShift on Azure: The easiest, fully managed OpenShift in the cloud - OpenShift on Azure makes it easier to set up and use with native Azure integration and leveraging the same Kubernetes technology that powers Azure Kubernetes Service. Microsoft and Red Hat jointly demonstrated the OpenShift on Azure offering, the first fully managed and easiest to use version of OpenShift in the cloud.

Spark + AI Summit: Data scientists and engineers put their trust in Microsoft Azure Databricks - At Spark+AI Summit 2018 in June, we’ll be showcasing Microsoft’s commitment to helping our customers drive analytics at scale and increase productivity with security, as well as the groundbreaking work of customers and partners that are using our technology to digitally transform their businesses. Learn more about our data and AI services on Azure, especially Azure Databricks.

The Azure Podcast

Customers and partners

Accelerating AI on the intelligent edge: Microsoft and Qualcomm create vision AI developer kit - At the Microsoft Build developer conference, we announced a partnership with Qualcomm, one of the largest mobile and IoT chipset manufacturers in the world, to jointly create a vision AI developer kit. This will empower Qualcomm’s latest AI hardware accelerators to deliver real-time AI on devices without the need for constant connectivity to the cloud or expensive machines.

Customer success stories with Azure Backup: Somerset County Council - Learn how United Kingdom’s Somerset County Council were able to improve their backup and restore efficiency and reduce their backup cost by using Azure Backup. Azure backup provided a simple and intuitive solution for their diverse workload backups and provides the agility to backup their ever-growing data to Azure.

Azure tips & tricks

Developer spotlight

Bring the power of serverless to your IoT application and compete for cash prizes - The Azure IoT on Serverless hackathon is an online, judged competition with cash prizes hosted on Devpost. All ideas are welcome, whether you want to work on that sensors-driven smart-home project you have been putting off, build a remote monitoring solution for a healthcare facility, create an intelligent system to streamline the manufacturing process of your production plant, or even create a self-healing robot wearing cool sunglasses.

Seven Properties of Highly Secure Devices - This paper from Microsoft Research identifies seven properties required in all highly secure devices, and describes an experiment working with a silicon partner to revise one of their microcontrollers to create a prototype, highly secure microcontroller.

Azure IoT Reference Architecture v2.0 - This 61-page document provides an overview of the recommended architecture and implementation technology choices for how to build Azure IoT applications. This architecture describes terminology, technology principles, common configuration environments, and composition of Azure IoT services.

Azure Friday

Azure Maps

In this seven-part series, Scott Hanselman explores Azure Maps with members of the team that build Azure Maps, which is now generally available and was formerly known as Azure Location-Based Services during preview.

Azure Friday #409 | Search in Azure Maps - Julie Kohler joins Scott Hanselman to discuss the core set of search APIs available in Azure Maps. In this episode, you'll learn how to do geocoding, reverse geocoding, search for Points of Interest and see an example using the JavaScript map control.

Azure Friday #410 | What's New for Search in Azure Maps - Julie Kohler joins Scott Hanselman to discuss the newest search APIs that have been added to Azure Maps. In this episode, you'll learn about doing a nearby search, searching within a given geometry and how to search along a route providing a maximum detour time.

Azure Friday #411 | Traffic in Azure Maps - Julie Kohler joins Scott Hanselman to discuss how to use the traffic APIs in Azure Maps. You will learn about how to surface the best in class traffic data coming from our partnership with TomTom using a set of simple REST APIs. Get up to date information on traffic flow as well as incidents.

Azure Friday #412 | Time Zones in Azure Maps - Julie Kohler joins Scott Hanselman to discuss how to use the set of time zone APIs available in Azure Maps. Learn about how this service is easy to use and how to get up to date information about time zones that change frequently around the world.

Azure Friday #413 | Designing with Azure Maps - Azure Maps were designed with the cloud customer in mind. Learn about the cartography design process as well as the robust data behind the curtains.

Azure Friday #414 | Routing with Azure Maps - Ricky Brundritt joins Scott Hanselman to discuss the routing capabilities in the Azure Maps Directions API. Azure Maps is a portfolio of geospatial services that include service APIs for Maps, Search, Routing, Traffic, and Time Zones. The portfolio of Azure OneAPI compliant services allows you to use familiar developer tools to quickly develop and scale solutions that integrate location information into your Azure solutions.

Azure Friday #415 | Visualizing Data with Azure Maps Web Map Control - Ricky Brundritt joins Scott Hanselman to discuss the rich data visualization capabilities in the Azure Maps web map control. Azure Maps is a portfolio of geospatial services that include service APIs for Maps, Search, Routing, Traffic, and Time Zones. The portfolio of Azure OneAPI compliant services allows you to use familiar developer tools to quickly develop and scale solutions that integrate location information into your Azure solutions.

Azure Friday at Build 2018

We left the studio and recorded a series of short episodes with experts from a cross-section of Microsoft Azure kiosks in the Expo Hall:

Kafka on HDInsight is Microsoft Azure’s managed Kafka cluster offering. It has generated huge customer interest and excitement since its general availability in December 2017.

Here are 10 great things about it:

1. Enterprise grade Kafka

HDInsight Kafka is an enterprise-grade managed cluster service giving you a 99.9 percent SLA. We take care of the heavy lifting for you, including operations, monitoring, security patching, and more. Running some of the largest Kafka clusters on the planet, we have the expertise and knowledge to prevent and quickly fix any issues.

2. Up and running in minutes

Create a full cluster in minutes, not days. Traditionally, it takes weeks to provision your own Kafka cluster on premises or on IaaS. With HDInsight, you get your own dedicated Kafka cluster within a few clicks.

3. Real open-source Kafka

With HDInsight, you get the true Apache Kafka engine running on your clusters with all its functionality. People love Kafka for its high throughput, lowest latency and cost-efficient scale. Along with the Producer and Consumer APIs, Kafka also has a rich feature set, such as compression of messages for an even higher throughput, configurable retention policy (including retention beyond 7 days and size based retention) and log compaction.

4. Use Kafka Streams for analytics

Apache Kafka is not just an ingestion engine, it is actually a distributed streaming platform with an amazing array of capabilities. With HDInsight, you get the Streams API, enabling users to filter and transform streams as they are ingested.

5. Use Kafka Connect

HDInsight supports the Kafka Connect API. Use community supported or custom connectors to receive from or send data to your favorite sources and sinks (such as IoTHub). With the Streams and the Connect APIs, customers can avoid having to manage and learn another technology, while also saving on costs as analytics can be collocated with your Kafka cluster.

6. Leverage the full power of Azure

You get all the managed cluster benefits, while still having full control of your cluster. You can install custom code on your clusters, isolate the cluster in your own custom VNET and apply rules to control the traffic flow into and out of the cluster.

7. Scale as you need

You get native integration with Azure Managed Disks, allowing you to separate compute and storage. You can scale your cluster within minutes to handle any throughput.

8. Enterprise grade management

HDInsight provides integration with Operations Management Suite so you don’t have to install your own custom solution. Features like Cluster self-healing ensure the highest-availability.

9. App marketplace

And you can easily amplify the productivity of your clusters by installing applications on the HDInsight Platform.

10. No limits

Last but not the least, there are no limits on the message size, number of consumer groups or throughput.

As you can see, Microsoft is deeply committed to Apache Kafka’s success. Click here to get started with HDInsight Kafka.

With Azure Data Factory (ADF) visual tools, we listened to your feedback and enabled a rich, interactive visual authoring and monitoring experience. It allows you to iteratively create, configure, test, deploy and monitor data integration pipelines without any friction. The main goal of the ADF visual tools is to allow you to be productive with ADF by getting pipelines up and running quickly without requiring to write a single line of code.

We continue to add new features to increase productivity and efficiency for both new and advanced users with intuitive experiences. You can get started by clicking the Author and Monitor tile in your provisioned v2 data factory blade.

Check out some of the exciting new features enabled with data factory visual tools since public preview (January 2018):

Latest data factory updates

Follow exciting new updates to the data factory service.

View Data Factory deployment region, and resource group. Then, switch to another data factory that you have access to.

Visual authoring

More data connectors

Ingest data at scale from more than 70 on-premises/cloud data sources in a serverless fashion.

New activities in toolbox

• Notebook Activity: Ingest data at scale using more than 70 on-premises/cloud data sources and prepare/transform the ingested data in Azure Databricks as a Notebook activity step in data factory pipelines.
• Filter Activity: Filter data ingested from more than 70 data sources.
• Execute SSIS Package: Execute SSIS packages on Azure SSIS Integration Runtime in your data factory.
• Look up Activity: Lookup activity now supports retrieving a dataset from any of 70+ ADF-supported data sources.

Azure Key Vault integration

Store credentials for your data stores and computes referred in Azure Data Factory pipelines in an Azure Key Vault. Simply create Azure Key Vault linked service and refer to the secret stored in the Key vault in your data factory pipelines.

Iterative development and debugging

Iteratively develop and debug your ETL/ELT pipelines with data factory visual tools. Perform test runs to debug your pipelines or put breakpoints to debug a portion of your pipeline.

View test run (debug) status on activity nodes

You can now view the the last test run status on activity nodes on the pipeline canvas.

Clone pipelines and activities

You can now clone an entire pipeline or an activity on the pipeline canvas. This will create an identical copy of the entire pipeline or an activity on the pipeline canvas including the settings.

New Resource Explorer actions

You can now expand/collapse all the resource explorer entities (pipelines, datasets) with a click of a button. You can also adjust the width of the ‘Resource Explorer’ by dragging it to the left/right.

View/edit Code for your data factory pipelines

You can now view and edit JSON for your data factory pipelines. Simply click the ‘Code’ icon to view your JSON, make edits directly to your JSON and click ‘Finish.’ You can then ‘Publish’ your changes to the data factory service.

View pending changes to be published to data factory

Add/edit/delete pipelines, triggers, datasets, linked services, integration runtimes and see the number of pending changes to be published to the data factory service.

Import data factory resources to any branch in your GIT repository

You can now choose the collaboration branch (generally ‘master’), create a new branch or use any existing branch to import your data factory resources while setting up the VSTS GIT repository. This is very useful in case you don’t have access to the collaboration branch and want the data factory resources imported in any other develop/feature branch.

Visual Monitoring

Monitor Copy real time progress

Click the ‘Details’ icon to view your copy activity continuous progress. Simply click ‘Refresh’ to get the latest statistics.

Alerts

Create alerts to be notified on different data factory metrics. For example: pipeline, activity, trigger failure runs. Clicking ‘Alerts’ will take you to the ‘Monitor’ tab in azure portal where you can create alerts on data factory metrics.

Metrics

Visualize your data factory metrics and see the pattern over days, months, and more in a simple graphical interface. Clicking ‘Metrics’ will take you to the ‘Monitor’ tab in azure portal where you can visualize your data factory metrics.

View run status of child pipelines

If your pipeline triggers other child pipelines using ‘Execute Pipeline’ activity in data factory, you can now view the status of child pipelines from the parent pipeline. Simply click the ‘Output’ icon under the ‘Actions’ column and click on the ‘pipelineRunId’ field.

Easily copy ‘runid’ for debugging purpose

You can now copy the ‘runid’ of your pipeline, activity runs easily for debugging purposes. Simply select and copy it in case you need to provide it to azure support for debugging purposes.

Our goal is to continue adding features and improve the usability of Data Factory tools. Get started building pipelines easily and quickly using Azure Data Factory. If you have any feature requests or want to provide feedback, please visit the Azure Data Factory forum.

Today we’re previewing the new Surface Hub 2, built from the ground up with Microsoft Teams in mind. More flexible and powerful than its predecessor, the Surface Hub 2 is designed to fit any workspace or work style. Head over to the Windows blog for all the details.

The post Unlocking the power of the team with Surface Hub 2 appeared first on Microsoft 365 Blog.

We are seeing more developers building and running their applications in the public cloud. In fact, companies are using multiple public clouds to run their applications. Our customers tell us that they choose to build applications in Azure because it’s easy to get started and that they have peace of mind knowing the services that their applications rely on will be available, reliable, and secure. Today, we are going to discuss how Azure Security Center’s Just-in-Time VM Access can help you secure virtual machines that are running your applications and code. 

Successful attacks on your virtual machines can create serious challenges for development. If a server is compromised, your source code could potentially be exposed, along with the proprietary algorithms or internal knowledge about the application. The pace of development can slow down because your team is focused on recovering from the attack instead of writing and reviewing code. Most importantly, an attack can affect your customers’ abilities to access your applications, impacting your brand and your business. Just-in-Time VM Access can help you reduce your exposure to attacks by limiting the amount of time management ports are open on the virtual machines running your code.

Just-in-Time VM Access can be found under Security Center’s advanced cloud defense features.

When you click on Just-in-Time VM Access, Security Center will automatically discover which virtual machines have Just-in-Time VM Access enabled. By default, it’s going to recommend that you block access to management ports, as those are most commonly attacked, but you can specify access to any port that you want to grant access to, the protocol for connecting, where you can connect from, and for how long.

To request access to a port on a virtual machine that has Just-in-Time VM Access enabled, you can visit the Configured tab of Just-in-Time VM Access or execute a PowerShell cmdlet. You can set permissions for certain users in your organization to only be able to do certain tasks. Even once access has been requested and granted, you can limit the amount of time this user can spend with the virtual machine.

Based on the rules set, you specify what port you need to access, where the request is going to come from, and the time range. The request cannot exceed the maximum time set. If the request complies with the rules set, you receive access to the virtual machine. Just-in-Time VM Access keeps the ports on VMs open for the bare minimum of time needed to complete a task and then it’s automatically closed, drastically reducing your available surface area for attack.

Just-in-Time VM Access is a part of Security Center’s Standard Tier, which is free for the first 60 days. To learn more about how you can set up Just-in-Time VM Access, visit our documentation.

¹Rightscale: Cloud Computing Trends 2017

Expanded intelligent answer coverage

 

Updated hotel booking experience

Hotel booking can be a pain, requiring you to do a bunch of searches and try several sites to ensure you’re getting the best deal.

Now on Bing you can simply search for hotels in a desired location, and when you click on one of the options you’ll see a full-page experience which lists options by price, location, rating, and availability by date. The experience also provides an aggregated view of this across popular third party booking sites like Expedia and Tripadvisor, so once you’re done browsing you can click straight through to booking via your preferred service. This experience is live in the US and the UK, where users will get results for both domestic and many international locations.

This experience is part of our ongoing investment to help you book hotels with less effort, so stay tuned for more exciting updates soon!


 

Updated weather results

 

Bing homepage contest

We value your feedback

We hope you’re as excited about these features as we are! As always, we truly value your perspective, and one of the best ways for you to provide it is via the Bing Insider Program, which gives our most passionate users the opportunity to provide feedback to Bing engineers and partner teams via monthly calls, Insider events, interactive feedback sessions, and more.

Please register at the Bing Insider site to be a Bing Insider. And even if you don’t join today, the website is chock-full of great content featuring our Insiders as well as new Bing features. We hope you enjoy the new site and come back often.

We heard from our customers that they wanted an even simpler onboarding experience to Azure Security Center. Today, we are excited to announce the general availability of Azure Security Center’s Cross Subscription Workspace Selection. This capability allows you to collect and monitor data in one location from virtual machines that run in different workspaces, subscriptions, and run queries. 

When first onboarding to Azure Security Center, you’ll need to start in our Data Collection tab to provision a monitoring agent onto Azure. The agent will allow you to monitor the security state of your hybrid cloud resources. As virtual machines are being spun up and down, and workload owners across your organization are creating new workloads and resources, you need to make sure these are protected at the time they are created.

By default, we put the data that the monitoring agent collects in a Log Analytics workspace, but we give you the flexibility to use another workspace if you are using it already for other management functions.

Today, when you select a workspace for your data to reside in, you’ll see all the workspaces across all your subscriptions available. Cross Subscription Workspace Selection allows you to collect data from virtual machines running in different subscriptions and store it in the workspace of your choice. This capability works for both virtual machines running on Linux and Windows.

Cross Subscription Workspace Selection a part of Azure Security Center’s Free Tier, making it easy for you to get started today. To learn more about how to set up this capability and other Azure Security Center features, visit our documentation. For any feedback or additional information contact ASC_Onboarding@microsoft.com.