ANN: Robot - Simulate keyboard and mouse events under X11 - Haskell-Cafe

ANN: Robot - Simulate keyboard and mouse events under X11

Chris Wong

23 Apr 2013 23 Apr '13

12:02 a.m.

I'm pleased to announce the initial release of Robot! Robot lets you send fake keyboard and mouse events, just like its namesake in Java. Only X11 systems are supported right now (via XTest), but Windows and Mac can be added later if anyone cares. ## Features + Simple API (only 9 functions and 3 types) + Few dependencies + Exception safe -- unlike with other libraries, all keys and buttons are released automatically when the robot terminates. `bracket` and `finally` are used where appropriate. ## Links Hackage: http://hackage.haskell.org/package/robot-1.0 Examples: https://github.com/lfairy/robot/tree/master/examples Happy hacking! Chris -- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

Show replies by date

Jeanne-Kamikaze

6 May 6 May

1:18 p.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

Looks like an interesting library. Will it be able to read pixels from a window at some point? On 23 April 2013 06:02, Chris Wong wrote:

...

I'm pleased to announce the initial release of Robot!

Robot lets you send fake keyboard and mouse events, just like its namesake in Java.

Only X11 systems are supported right now (via XTest), but Windows and Mac can be added later if anyone cares.

## Features

+ Simple API (only 9 functions and 3 types)

+ Few dependencies

+ Exception safe -- unlike with other libraries, all keys and buttons are released automatically when the robot terminates. `bracket` and `finally` are used where appropriate.

## Links

Hackage: http://hackage.haskell.org/package/robot-1.0

Examples: https://github.com/lfairy/robot/tree/master/examples

Happy hacking! Chris

-- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Niklas Hambüchen

8 May 8 May

12:47 p.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

Hi, I just started using your library to move my cursor. Is it possible that it ignores negative values in moveBy? In other words, I can only move the cursor into one direction.

Chris Wong

9:36 p.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

On Thu, May 9, 2013 at 4:47 AM, Niklas Hambüchen wrote:

...

Hi,

I just started using your library to move my cursor.

Is it possible that it ignores negative values in moveBy?

In other words, I can only move the cursor into one direction.

I did some research, and traced this to a bug in an old (1.6) version of xcb-proto [1]. The coordinates were declared incorrectly as Word16, instead of Int16 as they should have been. It's been fixed in xcb-proto since 1.7. I've cc'd Antoine Latter, the maintainer of XHB, about this bug. Once he uploads a new version of XHB, I'll be happy to fix it on my end. [1] http://cgit.freedesktop.org/~keithp/xcb-proto/commit/src/xtest.xml?id=f3ae97... On Tue, May 7, 2013 at 5:18 AM, Jeanne-Kamikaze wrote:

...

Looks like an interesting library. Will it be able to read pixels from a window at some point?

Not sure -- I have no idea how screen capturing works in X11. Calling gnome-screenshot should probably cover most use cases. Chris -- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

Tobias Dammers

9 May 9 May

5:37 a.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

On Thu, May 09, 2013 at 01:36:52PM +1200, Chris Wong wrote:

...

On Tue, May 7, 2013 at 5:18 AM, Jeanne-Kamikaze wrote:

...
Looks like an interesting library. Will it be able to read pixels from a window at some point?

Not sure -- I have no idea how screen capturing works in X11. Calling gnome-screenshot should probably cover most use cases.

I just took 2 minutes looking at the source code for scrot (which is about as simplistic a screen grabber as you'll find), and it looks fairly simple: it uses giblib, which provides a function called gib_imlib_create_image_from_drawable, which does exactly that. Not sure if Haskell bindings for giblib exist already though.

Chris Wong

12 May 12 May

4:02 a.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

On Thu, May 9, 2013 at 1:36 PM, Chris Wong wrote:

...

On Thu, May 9, 2013 at 4:47 AM, Niklas Hambüchen wrote:

...
Hi,

I just started using your library to move my cursor.

Is it possible that it ignores negative values in moveBy?

In other words, I can only move the cursor into one direction.

I did some research, and traced this to a bug in an old (1.6) version of xcb-proto [1]. The coordinates were declared incorrectly as Word16, instead of Int16 as they should have been. It's been fixed in xcb-proto since 1.7.

Okay, I've released a new version of Robot (1.0.1.1), that should fix this bug. Niklas: can you try it out please? Also, it turns out taking a screenshot is much easier than I thought. A single call to getImage returns a list of bytes, which happens to match exactly the internal structure used by JuicyPixels. I'll look deeper into this when I get the time. Chris -- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

Niklas Hambüchen

4:46 a.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

Yes, that works now. I have another problem though: I move the cursor at high resolution (128 Hz) and it seems that when robot issues a command to X, it disables (keyboard) state so far. This means that it's impossible for me to Ctrl-C my program: Only "c" is sent all the time, me pressing Ctrl seems to be reset with the next robot event. On Sun 12 May 2013 16:02:06 SGT, Chris Wong wrote:

...

On Thu, May 9, 2013 at 1:36 PM, Chris Wong wrote:

...
On Thu, May 9, 2013 at 4:47 AM, Niklas Hambüchen wrote:

...
Hi,

I just started using your library to move my cursor.

Is it possible that it ignores negative values in moveBy?

In other words, I can only move the cursor into one direction.

I did some research, and traced this to a bug in an old (1.6) version of xcb-proto [1]. The coordinates were declared incorrectly as Word16, instead of Int16 as they should have been. It's been fixed in xcb-proto since 1.7.

Okay, I've released a new version of Robot (1.0.1.1), that should fix this bug. Niklas: can you try it out please?

Also, it turns out taking a screenshot is much easier than I thought. A single call to getImage returns a list of bytes, which happens to match exactly the internal structure used by JuicyPixels. I'll look deeper into this when I get the time.

Chris

-- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Chris Wong

8:39 p.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

On Sun, May 12, 2013 at 8:46 PM, Niklas Hambüchen wrote:

...

Yes, that works now.

Excellent!

...

I have another problem though: I move the cursor at high resolution (128 Hz) and it seems that when robot issues a command to X, it disables (keyboard) state so far. This means that it's impossible for me to Ctrl-C my program: Only "c" is sent all the time, me pressing Ctrl seems to be reset with the next robot event.

Can you show me the code that triggers that behavior? -- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

Niklas Hambüchen

8:49 p.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

...

Can you show me the code that triggers that behavior?

It is basically Just connection <- connect forever $ do (x,y) <- getGyroMovement runRobotWithConnection (moveBy x y) connection

Chris Wong

13 May 13 May

8:25 p.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

Oh, I see now. I originally made the runRobot functions reset the input state when the Robot finished running. That worked well for my use case (testing GUIs), but as you have noticed, it causes unintuitive behavior when runRobot is called at a high frequency. In hindsight, that was a design flaw on my part: that resetting behavior should be specified explicitly, not attached unconditionally to every call to runRobot. I've removed the offending code, and released it as version 1.1. Hopefully I've ironed out the issues now :) On Mon, May 13, 2013 at 12:49 PM, Niklas Hambüchen wrote:

...

...
Can you show me the code that triggers that behavior?

It is basically

Just connection <- connect forever $ do (x,y) <- getGyroMovement runRobotWithConnection (moveBy x y) connection

-- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

Niklas Hambüchen

8:46 p.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

Awesome, that works very well, and it even made my program run faster / with less CPU. The reset functionality is useful, but I think optional is better. Did you remove it entirely or is it still available? On Tue 14 May 2013 08:25:04 SGT, Chris Wong wrote:

...

Oh, I see now. I originally made the runRobot functions reset the input state when the Robot finished running. That worked well for my use case (testing GUIs), but as you have noticed, it causes unintuitive behavior when runRobot is called at a high frequency.

In hindsight, that was a design flaw on my part: that resetting behavior should be specified explicitly, not attached unconditionally to every call to runRobot.

I've removed the offending code, and released it as version 1.1. Hopefully I've ironed out the issues now :)

On Mon, May 13, 2013 at 12:49 PM, Niklas Hambüchen wrote:

...
...
Can you show me the code that triggers that behavior?

It is basically

Just connection <- connect forever $ do (x,y) <- getGyroMovement runRobotWithConnection (moveBy x y) connection

-- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

Chris Wong

11:44 p.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

I removed the functionality because I didn't really see a use for it anymore. The `hold` and `tap` functions are already exception safe (thanks to `bracket`), and anyone who uses the unguarded `press` function probably wants to keep it held down anyway. Chris On Tue, May 14, 2013 at 12:46 PM, Niklas Hambüchen wrote:

...

Awesome, that works very well, and it even made my program run faster / with less CPU.

The reset functionality is useful, but I think optional is better. Did you remove it entirely or is it still available?

On Tue 14 May 2013 08:25:04 SGT, Chris Wong wrote:

...
Oh, I see now. I originally made the runRobot functions reset the input state when the Robot finished running. That worked well for my use case (testing GUIs), but as you have noticed, it causes unintuitive behavior when runRobot is called at a high frequency.

In hindsight, that was a design flaw on my part: that resetting behavior should be specified explicitly, not attached unconditionally to every call to runRobot.

I've removed the offending code, and released it as version 1.1. Hopefully I've ironed out the issues now :)

On Mon, May 13, 2013 at 12:49 PM, Niklas Hambüchen wrote:

...
...
Can you show me the code that triggers that behavior?

It is basically

Just connection <- connect forever $ do (x,y) <- getGyroMovement runRobotWithConnection (moveBy x y) connection

-- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

-- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

Niklas Hambüchen

14 May 14 May

2:21 a.m.

New subject: ANN: Robot - Simulate keyboard and mouse events under X11

Ah OK. Thank you again for the fast fixes! On Tue 14 May 2013 11:44:43 SGT, Chris Wong wrote:

...

I removed the functionality because I didn't really see a use for it anymore. The `hold` and `tap` functions are already exception safe (thanks to `bracket`), and anyone who uses the unguarded `press` function probably wants to keep it held down anyway.

Chris

On Tue, May 14, 2013 at 12:46 PM, Niklas Hambüchen wrote:

...
Awesome, that works very well, and it even made my program run faster / with less CPU.

The reset functionality is useful, but I think optional is better. Did you remove it entirely or is it still available?

On Tue 14 May 2013 08:25:04 SGT, Chris Wong wrote:

...
Oh, I see now. I originally made the runRobot functions reset the input state when the Robot finished running. That worked well for my use case (testing GUIs), but as you have noticed, it causes unintuitive behavior when runRobot is called at a high frequency.

In hindsight, that was a design flaw on my part: that resetting behavior should be specified explicitly, not attached unconditionally to every call to runRobot.

I've removed the offending code, and released it as version 1.1. Hopefully I've ironed out the issues now :)

On Mon, May 13, 2013 at 12:49 PM, Niklas Hambüchen wrote:

...
...
Can you show me the code that triggers that behavior?

It is basically

Just connection <- connect forever $ do (x,y) <- getGyroMovement runRobotWithConnection (moveBy x y) connection

-- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

-- Chris Wong, fixpoint conjurer e: lambda.fairy@gmail.com w: http://lfairy.github.io/

4390

Age (days ago)

4411

Last active (days ago)

List overview

Download

12 comments

4 participants

participants (4)

Chris Wong
Jeanne-Kamikaze
Niklas Hambüchen
Tobias Dammers