For agents

How a program on this computer, or a signed-in browser somewhere else, can drive a computer you own.

Only computers you own. Do not point this at a machine that is not yours. The switch that lets a program drive the mouse is off until someone turns it on at the computer.

Sign in to see the viewer link for each computer. The shape is /view/ plus the computer id.

Remote browser

Use the same signed-in session as a phone. The viewer page on the public site is https://speeddesk.app/view/ plus the computer id. The signaling socket is wss://speeddesk.app/signal?computer= plus that same id. The browser sends the site session cookie. Do not invent a second login.

After the socket opens, the site sends joined with iceServers. That list always has a STUN server. A relay entry is added only when this computer allows a relay and the operator has configured one. The relay password is not on this page.

The browser then makes a WebRTC offer. The computer answers. Candidates use webrtc_ice_candidate. Direct is tried first. Relay candidates are held for about 3 seconds. If direct connects, the relay is not used. If it does not, and a relay is configured, both sides release the relay candidates. The browser uses the TLS relay on port 443 so a blocked UDP port does not keep the picture waiting.

A relay is what lets a browser on another network succeed when both networks hide the machines. If the checkbox is off, or no relay is configured, a hidden computer stays unreachable. The failure text says which path was tried.

The picture is a video track on the WebRTC connection. It is not proxied through this website, and a screenshot is not uploaded here.

Coordinates

The picture is one rectangle: the captured screen. The origin is the top-left pixel. x grows to the right. y grows down. Both are fractions from 0 to 1 of that rectangle, not a drag distance and not a pixel count.

One move places the cursor on that point. Do not send a path of points. The center is 0.5, 0.5. A pixel on a 1920×1080 screen is x = pixelX / 1920 and y = pixelY / 1080.

The computer reports the same grid on the data channel. w and h are the pixel size of the rectangle. px and py are the cursor in those pixels. x and y are the fractions.

{ "t": "cur", "on": true, "x": 0.5, "y": 0.4, "px": 960, "py": 432, "w": 1920, "h": 1080 }
{ "t": "move", "x": 0.5, "y": 0.4 }

In the signed-in viewer there are three ways, and the script stays either way. A browser that cannot call page script uses the Jump box on the picture: choose Type a point, type the fractions, and press Jump. That box is ordinary page controls. A browser that can call page script uses speeddesk.jump(0.5, 0.4) and speeddesk.click(0.5, 0.4). Choose Page script in that same box to see the reminder. speeddesk.frame() returns w, h, and the current fractions. The corner of the picture shows the same numbers. A program on the computer itself uses POST /move and does not use the page.

A browser that cannot call page script uses the Jump box, or opens /view/ plus the computer id plus ?jump=0.5,0.4. That jumps once after the picture appears. ?agent=1 fits the picture to a 1280×800 window and asks for a smaller picture. The status line under the picture shows the fractions, the pixels, direct or relay, how long the picture took, and the round trip. It is not covered by the cursor.

Website API

This is for a program that is not a browser. It is off until “Let bots control this computer through the website” is ticked at http://127.0.0.1:18740 on that computer. That page then shows the website token, sent as Authorization: Bearer. The token is not in the download. The computer id is on that same page. While the box is off the website answers 403.

GET /api/desk/COMPUTER/snapshot?w=1280&q=80
POST /api/desk/COMPUTER/move
{ "x": 0.5, "y": 0.4 }
POST /api/desk/COMPUTER/click
{ "x": 0.44, "y": 0.83, "button": "left", "double": false }
POST /api/desk/COMPUTER/type
{ "text": "hello", "enter": false }
POST /api/desk/COMPUTER/key
{ "key": "Enter" }

One call instead of a picture

Use one call for the whole step.

A common task is one POST /batch, not a screenshot, a click, another screenshot, and then type. GET /help is the machine-readable list. GET /ui?scope=fg&max=40 is the focused window as lines id role "name" x,y,w,h in pixels. POST /ui/click uses that id. Ids last 30 seconds. GET /ocr reads words. GET /windows lists windows. POST /win/focus uses an id or a title. GET /wait?win=Notepad&t=5 waits instead of polling. GET /clip and POST /clip move text, at most 8000 characters. Failures are {"ok":false,"err":"..."} and the sentence says how to fix the call. There is no launch route. Allow programs stays off unless the computer turns it on. That switch does not cover typing. Keys and text can still open a program, including from the Start menu, the same way a person at the keyboard can.

POST /api/desk/COMPUTER/batch
{ "a": [ { "op": "type", "text": "hello" }, { "op": "key", "key": "enter" } ] }
GET /api/desk/COMPUTER/ui?scope=fg&q=Notepad&max=20
POST /api/desk/COMPUTER/ui/click
{ "id": "e3" }

None of these paths have a picture extension. The answer is JSON and Cache-Control is no-store. The snapshot JSON includes w, h, px, py, link, at, q, and image. image is a JPEG, base64, with the pointer drawn on it. w in the query is the widest side, at most 1280. q from 1 to 100 is the JPEG quality. Move only places the cursor. Click uses the same grid, then clicks. Leave x and y out of click to click where the cursor already is. button is left or right. Type does not add Enter unless enter is true, and it stops at 500 characters. Key is one key or a simple combo such as Ctrl+A. The body must be a JSON object under 64 KB. A wrong token is 401. Website control turned off is 403. Calls sent too quickly are 429. A snapshot the program could not take is 503, and the error is the program's own sentence. A computer that is not connected is also 503, with the error "That computer is not running."

Data channel

The phone creates an ordered data channel named modulon-data. The computer accepts that channel. Messages are JSON text. Coordinates x and y are fractions from 0 to 1 across the picture.

{ "t": "move", "x": 0.5, "y": 0.4 }
{ "t": "down", "x": 0.5, "y": 0.4, "b": 0 }
{ "t": "up", "x": 0.5, "y": 0.4, "b": 0 }
{ "t": "wheel", "x": 0.5, "y": 0.5, "dy": 1 }
{ "t": "key", "down": true, "code": "KeyA" }
{ "t": "q", "n": 2, "fps": 30 }

Local API

This is only for a program on the same computer. It does not use the website. The address is loopback, usually http://127.0.0.1:18742. The real address is shown at http://127.0.0.1:18740 when “Allow programs on this computer” is on. It is not 18730, 18740, or 18741.

Send Authorization: Bearer and the token from that local page. A missing or wrong token is 401. Turning the switch off stops the listener and forgets the token. The token is not on this website.

GET /health
GET /windows
GET /monitors
GET /screenshot
POST /move { "x": 0.5, "y": 0.4 }
POST /click { "x": 0.5, "y": 0.5, "button": 0 }
POST /type { "text": "hello" }
POST /key { "code": "KeyA", "down": true }

If direct fails and the relay checkbox is on, a remote browser still needs the operator’s relay. The local API does not. It only works on the computer itself.