Microsoft’s new Power Apps feature turns sketches into apps

Showcasing its investments in AI, Microsoft today announced Power Apps Express Design, a new feature in the company’s no-code/low-code Power Apps service that can convert downloaded PDF, PowerPoint, Figma, screenshots screenshot or a hand-drawn sketch into a working application (after a few manipulations). Unveiled during Build 2022, Microsoft said Express Design uses “industry-leading” AI to analyze input and produce functional application controls backed by data storage.

Express Design is the first major AI-powered feature to arrive in Power Apps after the launch of Power Fx, which leverages OpenAI’s GPT-3 to translate instructions such as “find products whose names begin with” children “” into formulas – and it taps into the growing no-code/low-code trend. According to Gartner, up to 70% of applications will be developed using no-code/low-code technology, including AI, by 2025. The company further projects that “citizen developers” ( i.e. those without a formal IT background) will begin to far exceed the number of traditional developers in large organizations within the year.

Microsoft has explained how Express Design works in a series of blog posts. Microsoft’s Azure Cognitive Services platform object detection models perform image recognition, mapping forms and figures in documents (including sketches) to constructs that exist in an application. Express Design can figure out what might be a button, grouping or text box and automatically generate an app based on those designs – or at least an approximation of an app.

Power Apps express design

Picture credits: Microsoft

Express Design does not everything hard work for you. After it parses a file, application components such as fields, text labels, date pickers, and buttons must be manually marked and connected to a database in Microsoft’s Dataverse service. . As with apps built the usual way in Power Apps, users can add additional screens (either manually or by converting new sketches) and upload images to customize the background, buttons, and other components. before publication.

Interestingly, Microsoft first experimented with turning sketches into code in 2018 with Sketch2Code, a research project that used AI to convert handwritten drawings into working HTML prototypes. Many others have explored the idea, including Airbnb and Alibaba, which have developed their own AI systems to translate interface sketches into product mockups and code.

On the business side, startup Zecoda uses AI to automatically transform design files into front-end code. But Express Design has Microsoft’s scale advantage.

“One of the great efforts we’ve made is to take it to the next level, to go from hundreds of millions of people who can use these tools to billions of people who can use these tools,” said Charles Lamanna, vice president of Microsoft. applications and enterprise platform, was quoted as saying in a blog post. “If we want everyone to be a developer, we can’t plan to teach everyone how to write Python or JavaScript code. It’s not possible. But it is possible if we create the right experiences and let’s introduce them to enough people who can click and drag and drop and use concepts you’re familiar with to create amazing solutions.”

Express Design is now available to all Power Apps customers.

Rosemary C. Kearney