r/javascript • • Nov 30 '16

Modern JS developer workflow makes me sad

Edit:

Wow, I would never have expected so many replies and advises. I sincerely thank you all!

I would like to highlight 3 tips that have helped me the most:

  • @sdihgusdoiu - Hard Source Webpack Plugin. This plugin was rarely given a mention in other links given what a difference it makes and how simple it is to set up compared to other solutions.
    My mental model: it calculates a hash for Webpack config and package.json. If neither a compilation unit nor the hashes changed, it reuses output of previous builds for that compilation unit.
    This results in a massive decrease in (re)build times. ~70s build became ~5s.
    I wish this would be built into Webpack itself, although I understand there are reasons why not.  
     

  • @acemarke - Definitely a subjective hint that many people will dislike: Use eval for source maps in Webpack
    Not only is it by far the fastest option for Webpack to produce, the source it emits is what the browser actually executes. No more brain-dead breakpoint jumps by Dev Tools. Now you are free to copy-paste and play around with code directly from the Dev Tools.
    Yes, it does show an ugly transpiled code instead of your beautified ES6 + JSX, but it sounds much worse than it really is. Much more reliable and predictable than inline-source-map.  
     

  • @Retsam19 - for frequently used modules consider exporting them as globals using Expose loader
    Being able to dynamically REPL require(modulePath) for any module seems not to be possible, although there are some Stack Overflow answers claiming it is. I couldn't get them working
     


Original Post:

This post is partly a rant, partly a cry for help, partly an inquiry about any tips and tricks that could improve our day-to-day work.

My company adopted a new, modern JS stack; all pretty standard:

  • React

  • Babel ES6 + JSX

  • Redux

  • Mocha

  • Webpack

  • React Router

  • lodash

  • along with their 1262 transitive dependencies (I'm not kidding here!)

As a result, my daily productivity plummeted. Honestly, I feel like x86 Assembly must have a friendlier set of developer tools.

A non-exhaustive list of issues:

  • We used to laugh at Java guys with their sluggish Maven, at C++ compilation times.
    Nowadays it takes Webpack longer to build a todo app than for a complex Java project to download all dependencies, clean project, compile all files, run all tests, create a package and upload it to repository. npm is even worse

  • So...Webpack's hot reload, the mythical feature getting universal praise.
    Apart from the fact that "hot" means "after 5 or more seconds", it comes with the worst of both worlds - sometimes it does work, sometimes it just doesn't; with no indication whatsoever whether it succeeded or not. So you spend 2 minutes figuring out why your fix didn't work, only to find out it hasn't been deployed in the first place. A colleague of mine stated: "I just don't trust hot reload. I always reload the entire page"

  • Webpack or Babel or something comes with a special, depression inducing feature: if you make a syntax error in your file, instead of displaying a clear error, it loads some random cached build from half a day ago

  • my employer can definitely find a better use for its money than me tracing build bugs of some deeply nested transitive 3rd party library that whimsically has changed it's AMD/UMD exports

  • so...source maps. I can see my original sources in Developer Tools! Usually, that is. So, let's try to add a debug breakpoint to my code!
    If you are lucky, it will stay on the line you clicked. Other times, it jumps a few lines up or down. Occasionally, it jumps to the bottom of the file. When developer tools sense that you haven't been frustrated for full 10 minutes, the breakpoint jumps to a 200kLoC app.js file, which freezes the entire Dev Tools and Chrome with it. Now, don't close Chrome at this point! If you do so, the next time you open Dev Tools, it will try to reopen the last session and freeze again. Just wait patiently for 2 minutes till Dev Tools finishes parsing the file, grab a cup of coffee. Actually, an antidepressant might be a better choice.

  • you won a lottery and Dev Tools really did stop at your breakpoint. Congrats,. Now, let's inspect some expression. Uncaught ReferenceError: X is not defined. What? Well, Babel thinks the best way to compile this piece of code:
    moment.isDate(...)
    is to translate it to:
    _moment2.default.isDate(...)
    This is done to every ES6 import. And yes, that "2" in _moment2 can and will change to "3" or just disappear, at any point.

  • Let's write some tests. Hm...they failed, let's see what's happening. Well, Chrome executes your test before it loads your sourcemaps, so it totally ignores your breakpoints. Remember that 200kLoC app.js file? That's your best bet now.

  • One of the best features of Javascript was an interactive REPL console in the browser. But that's so 2014! With Webpack/Babel, it's just not possible to get a reference to a particular module from Dev Tools Console. By design. Because best practices, man!

  • On the other hand, we have the 8th Wonder of the World, a time-travelling debugger for Redux. Last time I productively used it was never.

 

Is this...normal?  

How do you cope it?  

How is this clusterfuck touted as a state of the art in Javascript?  

Did you also become alergic to the phrase

makes it easy to reason about

 

516 Upvotes

292 comments sorted by

View all comments

24

u/perestroika12 Nov 30 '16 edited Nov 30 '16

Have you ever tried another language? My maven (java/scala) build times can be 10 minutes for a full rebuild after I make a change, and that's just compile time. Then you have to run jetty to actually spin up the application (rest api in this case) which is another minute or two. Not saying it's okay just saying long compile times are not just js or web pack and usually is determined by the size and scope of the application.

No language is magically fast at 200k lines of code. Sounds like you moved from a small company to a large one and are now dealing with what it means to work in the same repo as 30 other devs.

8

u/[deleted] Nov 30 '16

That was the best back in my C++ days. "Oh dear, I changed this header file and now I'm going to be compiling for the next 20 minutes. Guess I'll have to take a break." It was great for those days where you just couldn't muster any kind of motivation.

2

u/perestroika12 Nov 30 '16

Scala is unfortunately sort of smart in that regard, between SBT and zinc you can incrementally compile. Intellij also does some hot reloading of classes so in most cases you don't even need to build. I would say my average build time is around 2-3 min. But when some large changes get pushed out, yeah, looking at a coffee run at least.

There's also this gem: http://zeroturnaround.com/software/jrebel/

I would assume wepack has the ability to incrementally compile but have not looked into it.

1

u/acemarke Nov 30 '16

Yeah, the Webpack DevServer (which internally uses the webpack-dev-middleware plugin) will watch files on disk and incrementally recompile whatever actually changed. Incremental builds are plenty fast. Pasting from my current dev server console:

webpack building...
webpack built 4e2fd1a32e44c9c6ffbd in 1950ms
Hash: 4e2fd1a32e44c9c6ffbd
Time: 1950ms

Takes time from there to push the changes up to the listener in the client and reload things, but even though that got a lot slower after I did some codebase rearchitecting, it's still only like 20 seconds. (It was like 4-5 seconds total, but my rearchitecting seems to have entangled dependencies a bit more than they used to be because I'm using index.js files to re-export things from folders. Still trying to sort that out.) In any case, a 20-second reload time keeping all of my app state intact is still way better than a several-minute complete app recompile, plus having to log out and back in.

1

u/MachaHack Dec 01 '16

It's okay. I have a project where any change to the gradle build scripts forces you to re-import them for the IDE to work, and on IntelliJ that means 10 minutes of my Mac going full throttle.

4

u/marosurbanec Dec 01 '16
mvn clean compile tomcat7:run

On a Java codebase of 70kloc, takes about 15 seconds. About half of the time spent obtaining a pool of DB connections = waiting for an external resource.

Babel/React stack - 16kloc, takes about a minute.

1

u/perestroika12 Dec 01 '16 edited Dec 01 '16

You shouldn't need to compile the entire project every time you make a change. Webpack has a fair amount of optimizations that you can use: https://webpack.github.io/docs/build-performance.html

Like any compiled language/framework a full rebuild is overkill.

2

u/davertron Nov 30 '16

Yeah, the op's experience is exactly the opposite of mine. We have a fairly large single page app, and it still kicks the shit out of any of our Java apps as far as compile/start-up times. I have no idea what kind of apps they have for the backend (micro-services maybe ;P) but before our single page app I worked in Grails and Java and I don't miss it one bit.

1

u/lechatsportif Dec 01 '16

10 minutes in any java build is a huge problem somewhere in your module or your using gradle. i was running a mixed scalajs and scala build on sbt incremental and annoyes it was taking 10 secs like 2 years ago. also, being statically analyzed java rebuilds arent needed as frequently.

1

u/mhd Dec 01 '16

And I've had C++ code bases that took literally hours. But that really shouldn't be the yardstick here. Compare it to interpreted languages, where setup times are usually quite quick. PHP is basically sold on this principle alone.

And 200k lines would've been no issue for Turbo Pascal on my ancient 286.

0

u/WatchDogx Nov 30 '16

If you use Scala as a benchmark for compile times then everything seems super fast.