UI Automation

UI automation is the mechanism underneath every desktop flow action that clicks, types into, or reads from a control: rather than clicking raw screen coordinates, Power Automate Desktop builds a hierarchical 'UI element tree' for a target application, similar to a DOM tree for a webpage, and each action stores a 'selector' describing the path from the root window down to the specific control. This means the same 'Click' action keeps working even if the window is resized, moved, or repositioned on screen, because the selector resolves by the control's place in the hierarchy and its properties (name, automation ID, class) rather than by fixed x/y coordinates. The UI Explorer tool in PAD lets you hover over any control in a running application to inspect its full selector path and individual properties before an action is even recorded, which is essential for building automation against applications the recorder alone struggles to capture cleanly.

Raw recorded selectors often embed brittle attributes like a numeric index ('Instance = 3') that break the moment a list reorders or a new row is inserted; the selector editor lets you replace index-based matching with a stable attribute such as 'Name' or 'AutomationId' when the target application exposes one, and you can mark individual attributes as 'variable' so a single selector can dynamically match different rows or tabs at runtime by injecting a Power Automate variable into that attribute. For grid-like UIs where no attribute uniquely identifies a row, an 'anchor' element, a nearby stable control used as a relative reference point, lets you say 'the button next to the row containing this exact order number' instead of relying on a fragile absolute index. PAD's fuzzy selector matching can also tolerate minor attribute drift (for example, matching on partial text) when exact matching is too strict for an application whose labels change slightly between versions.

When an application renders through a technology PAD can't fully inspect, such as certain Java Swing apps, some game engines, or heavily virtualized Citrix sessions, selector-based targeting may fail entirely, and PAD falls back to coordinate-based actions like 'Mouse click' at a fixed screen position or relative-to-window position. Relative positioning (offset from a window's top-left corner rather than the absolute screen) is more robust than absolute screen coordinates because it survives the window being moved, though it still breaks if the window is resized or the target's layout shifts internally. For the most demanding cases, PAD's 'Get details of window' plus custom offset math, or scripting actions calling into UI Automation APIs directly via a .NET/PowerShell script, provide an escape hatch when no built-in action reliably targets the control.