r/Python • • Nov 28 '20

Intermediate Showcase "Sillynium" - Auto generate working Python Selenium scripts by drawing coloured rectangles around web elements on a page that you want to automate

TLDR: 60 sec video here:https://streamable.com/1y17lt

TLDW: When I want to automate actions on a site, such as logins or forms, or scraping pages with JavaScript, I manually inspect each web element I need and begin writing a script.

I'm lazy though and had the idea to automate the creation of these automation/testing scripts. And thus "Sillynium" was born.

Sillynium allows you to draw boxes around the elements you require on a page and automatically gets the element and its properties for you. Different coloured rectangles tell the program you want to do different things. Eg: Red means "Input text", Green means "Click button".

When you've finished drawing boxes, a working python selenium script is generated. If you run that script, it does exactly what you told it to do! All this without you writing any code, and losing time.

Working 60 second video link of program in action: https://streamable.com/1y17lt

I made this project over the course of a day as a quick and dirty proof of concept. Its very flawed in its current iteration (taking a screenshot of webpage), so V2 I'm trying to figure out how to draw directly to the browser. Critique it and rip it to shreds. I'm aware of browser recorders btw so I know those exist, this is my twist on automation

Edit: I was not prepared for the amount of interest in this little proof of concept! So thank you all very much for your time you’ve made my day :)

In the interest of transparency and to answer your questions about the code and my plans:

I originally thought about monetisation (web extension etc) after finishing this POC (**sigh i know I’m a greedy bastard) but now I’m in 2 minds about it!

On the one hand I want to make it free for personal use, on the other hand I’d like to potentially charge for commercial use (once its finished).

Final edit 2 The community has spoken, I WILL BE RELEASING THIS VERSION AS OPEN SOURCE! I did some light refactoring, and have added it to a repo which will be shared in a follow up. It shows the POC and the community is welcome to build on the original idea :)

I will create a follow up post with links to repo once I’ve created it, this will be my first project on GitHub so try be nice ;) pretty nervous about sharing my code :D

Thank you all for your patience, interest, and awesome discussions

1.5k Upvotes

171 comments sorted by

View all comments

2

u/ravepeacefully Nov 28 '20

What’s better about this than scrapy?

2

u/[deleted] Nov 28 '20

Hey rave I haven’t used scrapy personally, though i’ve heard the learning curve can be initially quite steep and requires additional envs and installs. I guess this simplifies processes alot, and since we are visual creatures it works with that!

2

u/ravepeacefully Nov 28 '20

Ahh ok very cool but very similar to scrapy, you should check out theirs and use their ideas. Their founder is doing quite well in the entrepreneurial world

2

u/shaz55 Nov 28 '20

Would assume they are doing the same with this post!