Improvements in Working with Large Data Sources

Business apps typically need to connect to large data sources and schemas. That’s why in the Office Developer Tools for Visual Studio 2013 – March 2014 Update we've done a lot of work to improve the experience for working with large data models in both LightSwitch projects and Cloud Business Apps. In addition to improving designer performance, we've also added new features to the Attach Data Source Wizard and Entity Designer and support for complex types in OData feeds. In this post we’ll discuss each of these improvements.

If you don’t have the latest free update, download the Office Developer Tools for Visual Studio 2013 – March 2014 Update now.

Data Attach Improvements

We’ve added some new features to the Attach Data Source Wizard that make it easier to use with large, enterprise data sets and help you discover the entities within them. First off: the window is resizable now! This allows you to see more of your entities at once.

We’ve also revamped the “Choose your Items” page where you choose what entities you want to use. It now has three panes: the selection pane, the object pane, and the metadata pane.

Selection Pane

The entities in the data source are now organized into categories shown in the selection pane. Depending on what type of data source you are attaching, these categories change. For instance, if you’re attaching to a SQL database, you can choose between tables and views.  In the example above, we’re attaching to SharePoint, so the categories are lists and document libraries.  You can also narrow your view down to only your selected items.

Object Pane

The object pane contains the list of items you selected. The object pane sorts entity properties so that primary keys are first, with other properties listed in alphabetical order.  It also shows related entities to help you discover the meaning of the data through relationships. The object pane also supports type-ahead, so if you are looking for a specific schema item, you can just start typing an entity or property name and the wizard will move the selection to the next matching entry.

Metadata Pane

The metadata pane on the far right provides additional information for the currently highlighted object in the selection pane. This additional information varies depending on the object type and the data source type. For instance, data sources like SAP, SharePoint and other OData services will provide a hyperlink to browse the actual data.

Search Box

We’ve also added a search box to help you search schema items based on the name. When you enter a value, the entire list is searched and a new Search Results node will appear in the selection pane.

All of these improvements are geared toward helping you make sense of large, possibly unfamiliar, data sources.

Entity Designer Improvements

The Entity Designer (a.k.a Data Designer) has also gone through some improvements to help you work with large schemas. When you attach to a large, complex data source, you may notice some changes in the Entity Designer, particularly if the entity you are viewing has a lot of relationships. We wanted to make sure that the entity in view would display the maximum number of properties possible on the design surface so it will not shrink in size when there are a large number of direct relationships. Apart from the visual changes to the relationship lines, you can now sort properties on the entity itself.

Let us show you what we mean. This is how the Entity Designer would look in previous releases if you were attaching to an entity with many relationships (the following schema was chosen specifically to illustrate the point – we hope you don’t have to deal with something like this! :-)):

This is how the Entity Designer would now look:

Notice that we strive to display as many properties as possible in the entity in view, making it easier for you to realize the data model.

Just like before, to view the related entities, you can either double-click a related entity name to see the related entity’s properties, or double-click the relationship line to bring up the Relationship dialog where you can see the navigation property. In order to save even more space in the designer, any navigation property that isn’t currently within the viewport will connect to the top or bottom of the Entity Designer, depending on where it’s currently placed. The relationship lines that are not visible within the viewport will converge into a single horizontal and vertical line.

If you want to see the cardinality for a converged line, just select the relationship line from the related entity and the relationship’s navigation property will scroll into view.

Multi-select & Reordering Properties

We also added a couple other nice touches to the designer to help you model data much quicker. You can now select multiple entity properties and edit their common properties at the same time in the properties window. For instance, this comes in handy when you want to set multiple properties like “Display by Default”.

Also, we now allow you to alphabetically sort properties. When an entity has a large amount of properties, sometimes it’s hard to find them. Using the re-ordering feature on the entity, you can sort the properties in ascending or descending order. You can either perform sorting on few selected properties or on all the properties. When you reorder properties on an entity, this dictates the order they are shown by default on screens.

Note: A property which is part of the primary key will always be shown at the top, irrespective of the sorting order selected. However, they can be manually moved up/down by simply dragging and dropping the property at the desired position on the entity grid or by selecting ‘move up/ move down’ from the context menu.

Complex Type Support

Along with the improvements to the Attach Data Source Wizard and Entity Designer, this release also introduces support for complex types in OData feeds. This can be particularly helpful for building apps with complex data sources like SAP.

What Is a Complex Type?

In OData, a complex type is a composite data type consisting of zero or more properties. Entities can then define properties whose type is one of these complex types. Complex types typically represent a set of related information; they are frequently used in enterprise data sources, such as SAP, as they provide a way to organize large data schemas.

Let’s take a look at an example. Here is an excerpt from the SAP data source “GWDEMO”, which you can find on this list of SAP sample feeds.

"Address">"Street" sap:label="Street" MaxLength="60" Type="Edm.String"/>"City" sap:label="City" MaxLength="40" Type="Edm.String"/>"AddressTypeCode" sap:label="Address Type" MaxLength="2"  Type="Edm.String"/>"PostalCode" sap:label="Postal Code" MaxLength="10" Type="Edm.String"/>"CountryText" sap:label="Country" MaxLength="60" Type="Edm.String"/>"Building" sap:label="Building" MaxLength="10" Type="Edm.String"/>"AddressTypeText" sap:label="Text" MaxLength="60" Type="Edm.String"/>"CountryCode" sap:label="Country" MaxLength="3" Type="Edm.String"/>"AddressKey" sap:label="Address ID" MaxLength="32" Type="Edm.String"/></ComplexType>

This complex type represents an address, and it contains 9 string properties representing the components of the address. The feed then has an entity named “BusinessPartner” which contains a property named “Address” of this type. (Note that the property’s name does not have to match the complex type’s name; for example, an entity representing a person could have properties named “HomeAddress” and “WorkAddress” of the Address type.)

You can read more about complex types in the OData documentation.

Using Complex Types

When an entity with a complex type property is imported, the property is expanded (or “flattened”) to its primitive type components, which are then treated just like any other primitive type entity properties. The name of each such property is of the form “_”. For example, the “Address” property on BusinessPartner mentioned above is flattened to “Address_Street”, “Address_City”, etc., which are each string properties.

Here is how that looks in the Attach Data Source Wizard:

If we go ahead and import this entity, we can see that each “Address” property is of the type String. The app can then make use of these properties just like the other String properties. Previously, the “Address” property would not be exposed. Now, we support all complex type entity properties except nullable properties and properties whose complex type inherits from another complex type.

Wrap Up

With these improvements, attaching to and working with large or unfamiliar data sources should become a lot easier. The Data Attach wizard helps you find the entities you really want. The Entity Designer will no longer look cluttered and will display information in a more organized manner. In addition, your app will now be able to work with more complex types and schemas.

We hope you like the new features in this release. If you have feedback, leave a note below or ask a question in our forum, we’re there to help!

• Bilal Waheed, Software Development Engineer, Cloud Business Apps
• Chris Rummel, Software Development Engineer in Test, Cloud Business Apps
• Nate Amundson, Software Development Engineer, Cloud Business Apps
• Rohit Agrawal, Software Development Engineer in Test, Cloud Business Apps