r/javahelp • u/sellithy • Nov 11 '20
Help With Creating an Automating GUI Personal Project
I want to create a program that takes and analyzes screenshots, and simulates mouse clicks/movments. The inspiration came from Code Bullet and his " X Perfect AI" videos.
The issue is that the recent videos that I watched use Python, specifically the PyAutoGUI module. I personally do not prefer Python and I am much more comfortable with Java so I want to create my project in Java.
I understand that a Java.awt.Robot class exists and might serve what I want. However, this class is old (my feeling at least) and not pleasant to use.
My question is: does a (preferably high-level) Java library exist to interface with mouse and keyboard?
I am also open to a change in my strategy. Maybe it is better to only do part of the project in Java and use something like AutoHotkey. Any input is appreciated.
P.S: Not really related to the issue at hand, but I am actually planning to use Kotlin for this project as I wanted to learn the language. Does Kotlin provide more options for what I want to accomplish?
2
u/Ambrotus Nov 11 '20
I've been eyeing http://sikulix.com/quickstart/ for a long time, just havent gotten around to using it.
1
u/sellithy Nov 11 '20 edited Nov 11 '20
Yeah this looks promising. I remember stumbling upon it when searching about automation with Python. Did not realize that it also works with Java.
2
u/dionthorn this.isAPro=false; this.helping=true; Nov 11 '20
You might be interested in this repo: https://github.com/kwhat/jnativehook
It provides native level hooks for global listeners.
2
u/sellithy Nov 11 '20
This is kind of the reverse of what I want but I will definitely keep it in mind.
1
u/Ostricker Nov 11 '20
I did a quick search and didnt found much. Well seems everyone that needs Robot just creates wrapper for their use case so I guess.. do the same?
1
u/sellithy Nov 11 '20 edited Nov 11 '20
Yeah it is really sad. I found a lot of options for Python but Java was super limited. Most answers refrenced Robot class, which is obnoxious to use. Writing my own library is what I want to avoid. I want to automate in this project, not build a library.
1
u/ElFeesho Nov 11 '20
Kotlin does make it a lot easier to make old crusty APIs more palatable.
A trick you can use, is using extension functions to make the Robot class have the easy to use API you want, but, it's bad practice to write extension functions that don't extend the abstraction of the class you're writing the extension function for (for example adding a "isValidCreditCardNumber()" method to the String class.)
The best thing to do though, is to just create your own abstraction instead. If you're building a bespoke game manipulation thing for say snake, then you'd want something like a SnakeRemote maybe that would have the methods on it you'd wish a real API would have.
1
u/sellithy Nov 11 '20
Thank you for the advice. Is there an article or guide that explains using extension methods properly. I am currently doing the Koans FYI.
1
u/ElFeesho Nov 12 '20
If this is a throwaway project, it'd be good practice to have a go and feel any pain you encounter by using extension functions first hand.
Personally I find trying things out myself (in a small safe way, e.g on a throwaway project) gives me a better understanding of concepts and first hand experience I can rely on when discussing pros and cons of an implementation approach.
Right now I wouldn't worry too much about it, you're at the very early stages of learning kotlin, so have fun and just see what the language can do for you.
A lot of medium articles wash up on /r/kotlin and I think most of them contain good advice.
Good luck! Don't be afraid to share what you make on GitHub and keep us posted!
•
u/AutoModerator Nov 11 '20
Please ensure that:
Your code is properly formatted as code block - see the sidebar (About on mobile) for instructions
You include any and all error messages in full
You ask clear questions
You demonstrate effort in solving your question/problem - plain posting your assignments is forbidden (and such posts will be removed) as is asking for or giving solutions.
Trying to solve problems on your own is a very important skill. Also, see Learn to help yourself in the sidebar
If any of the above points is not met, your post can and will be removed without further warning.
Code is to be formatted as code block (old reddit: empty line before the code, each code line indented by 4 spaces, new reddit: https://imgur.com/a/fgoFFis) or linked via an external code hoster, like pastebin.com, github gist, github, bitbucket, gitlab, etc.
Please, do not use triple backticks (```) as they will only render properly on new reddit, not on old reddit.
Code blocks look like this:
You do not need to repost unless your post has been removed by a moderator. Just use the edit function of reddit to make sure your post complies with the above.
If your post has remained in violation of these rules for a prolonged period of time (at least an hour), a moderator may remove it at their discretion. In this case, they will comment with an explanation on why it has been removed, and you will be required to resubmit the entire post following the proper procedures.
To potential helpers
Please, do not help if any of the above points are not met, rather report the post. We are trying to improve the quality of posts here. In helping people who can't be bothered to comply with the above points, you are doing the community a disservice.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.