During the development of Internet Explorer 11, we proposed and implemented a new set of IME APIs to the Web Applications Working Group for a better input experience for East Asian users. These discussions resulted in changes to the Editor’s Draft of the W3C IME API. This API enables developers to build a tightly integrated experience between the IME that East Asian users rely on to type in their local languages and the UI that developers create to assist input, for example a search suggestion list that dynamically updates based on the keywords that the user has typed.
Avoid UI Occlusion
With IE11, developers can avoid the IME window occluding the part of the UI that is expected to be fully visible while the user is typing.
IME converts Roman alphabets into local characters. There are thousands of characters in East Asian languages. Often the same Roman input maps to several different East Asian words, in which case the user could choose from a list of candidates provided by the IME. The candidates UI is usually a floating window positioned close to the input box, and is not part of the DOM tree, thus JavaScript code doesn’t have a direct access to its size and position.
Example of the IME candidate window when typing Simplified Chinese
Many Web sites show a custom UI below the input box as a form of auto-complete. For example, a list of city names that continues narrowing down based on what the user has typed.
Example of an input suggestion UI
Since the IME candidate window is also below the input box by default, the two UIs overlap with each other, making it difficult for the user to see or interact with them.
Input suggestion UI occluded by the IME candidate window UI
Using the new IME API in IE11, developers can detect the opening of the IME candidate window by listening to MSCandidateWindowShow event, then call getCandidateWindowClientRect() function to find out where the candidate window is and position the suggestion UI away from it:
var context = document.getElementById("mySearchBox").msGetInputContext();
context.addEventListener("MSCandidateWindowShow", candidateWindowShowHandler);
function candidateWindowShowHandler(e) {
var imeRect = context.getCandidateWindowClientRect();
var suggestionRect = document.getElementById("mySuggestionList").getBoundingClientRect();
// Check if the two rects intersect, and position them away from each other.
}When the IME candidate window changes position or closes, it fires MSCandidateWindowUpdate or MSCandidateWindowHide events. Developers could listen to them and shift the suggestion UI accordingly.
For a more integrated look, developers can apply the new -ms-ime-align: after CSS property on the textbox to tell the IME to adjust its width and dock below the textbox.
The IME API Test Drive where the IME candidate window doesn’t occlude the Web site’s suggestion UI
Earlier Suggestions
Before IE11, developers had to wait until the user finished composing an East Asian character inside the IME to know what the new character would be. In IE11, the list of candidates inside the IME is also exposed to developers while the user is composing, allowing the site to generate suggestions much earlier.
For example, to type the city name of Shanghai in Chinese (“上海”), the user has to type “shang” and hit the space key to get the first Chinese character “上” into the textbox. The suggestion logic of the site couldn’t begin until this sequence was complete. Using the new IME API in IE11, the site gets the list of IME candidates by calling the new getCompositionAlternatives() function as soon as the user has typed the first “s” character; the character “上” is returned among several other candidates. By matching the IME candidate list to the list of city names, the site can find out that the user may be looking for “上海,” and show that in the suggestion UI, so the user could finish the input by only typing “s” and tapping on the suggestion, saving many keystrokes.
var searchBox = document.getElementById("mySearchBox");
var context = searchBox.msGetInputContext();
var compositionAlternates = context.getCompositionAlternatives();
var searchStrings = new Array();
// Add the user’s raw input as one search string.
searchStrings.push( searchBox.value.toLowerCase().trim() );
for (var i = 0; i < compositionAlternates.length; i ++)
{
// Merge the alternate with the string that is already in the search box.
var searchString = searchBox.value.substr(0, context.compositionStartOffset);
searchString += compositionAlternates[i];
searchString += searchBox.value.substr(context.compositionEndOffset);
searchStrings.push(searchString);
}
// Find out which string matches the beginning of a city name, and put that city name into the suggestion UI.
matchToCityNames(searchStrings);
Using getCompositionAlternatives() in IME API to provide earlier suggestions
Summary
With Internet Explorer 11, developers can enable an IME-aware input experience in their Web-based applications and sites where the IME collaborates tightly with the rest of the page and provides East Asian users a fast and fluid experience, especially in typing suggestion scenarios.
Please note that this API relies on some Windows 8.1 features of the Text Services Framework Interfaces so it is not fully functional with IE11 on Windows 7. It also requires support from the IME. At the time of writing this blog, only the in-box Chinese Simplified/Traditional IMEs in the desktop, and the in-box Chinese Simplified/Chinese Traditional/Japanese/Korean IMEs in the Modern UI environment have a full support of this API.
Please try out the IME API Test Drive on Windows 8.1 and start using the new APIs in your sites to support these scenarios for East Asian users. We look forward to your feedback through Connect.
Jianfeng Lin
Program Manager, Internet Explorer