r/java • u/RockLeeOfTheMounties • Jan 09 '20
Serverless Web Scraping with AWS Lambda
https://www.scrapingbee.com/blog/serverless-web-scraping-with-aws-lambda-and-java/2
u/zombays Jan 09 '20
A bit off-topic but are there possible good alternatives to AWS Lambda and Google App Engine? What are the pros and cons of using them?
8
u/newcomputer1990 Jan 09 '20
Microsoft Azure is pretty much all of your options besides the ones you listed
6
u/cogman10 Jan 09 '20
The closest I've seen is running your own FAAS stack.
For example, with OpenFAAS
https://www.openfaas.com/The pros and cons for AWS and the other platforms is pretty much the same. You are coupling with their platform. That means that Google/amazon/microsoft get to figure out how much they want to charge you and only market pressure really stops them from royally screwing you over.
pro - You don't have to really figure out where/when/what/how your script gets run. The providers do that for you.
con - Once you buy in, it is hard to "buy out" so to speak.pro - really easy to get up and running
con - The why's and how's of how your app runs are obfuscated away. A script could run in 1second or 10 depending on what needs to happen in the background to get things setup.1
1
Jan 09 '20
As a con, I found it not to easy running it locally Dir ng development and writing integration tests.
0
Jan 09 '20
[deleted]
0
Jan 09 '20
[deleted]
1
Jan 10 '20
a toilet seat is a toilet seat, it doesnt really matter which one you use. Is that what he wants to say?
1
u/Kango_V Feb 03 '20
Just published a Java function to Lambda.
It was written using Micronaut and compiled to a Linux native binary using GraalVM. I has a call latency of around 60ms.
Micronaut has an aws-lambda dependency which makes it trivially easy.
It is sooo fast.
0
-1
u/interactionjackson Jan 10 '20
java has gnarly start up times and i would t want a distributed crawler to be burdened by Java.
If this gets enough upvotes I will publish a crawler as a SAM template
6
u/lovett1991 Jan 09 '20
Did not realise you could use java in serverless! Thought it was for all these fancy JavaScript people.