r/javascript • • Nov 30 '16

Modern JS developer workflow makes me sad

Edit:

Wow, I would never have expected so many replies and advises. I sincerely thank you all!

I would like to highlight 3 tips that have helped me the most:

  • @sdihgusdoiu - Hard Source Webpack Plugin. This plugin was rarely given a mention in other links given what a difference it makes and how simple it is to set up compared to other solutions.
    My mental model: it calculates a hash for Webpack config and package.json. If neither a compilation unit nor the hashes changed, it reuses output of previous builds for that compilation unit.
    This results in a massive decrease in (re)build times. ~70s build became ~5s.
    I wish this would be built into Webpack itself, although I understand there are reasons why not.  
     

  • @acemarke - Definitely a subjective hint that many people will dislike: Use eval for source maps in Webpack
    Not only is it by far the fastest option for Webpack to produce, the source it emits is what the browser actually executes. No more brain-dead breakpoint jumps by Dev Tools. Now you are free to copy-paste and play around with code directly from the Dev Tools.
    Yes, it does show an ugly transpiled code instead of your beautified ES6 + JSX, but it sounds much worse than it really is. Much more reliable and predictable than inline-source-map.  
     

  • @Retsam19 - for frequently used modules consider exporting them as globals using Expose loader
    Being able to dynamically REPL require(modulePath) for any module seems not to be possible, although there are some Stack Overflow answers claiming it is. I couldn't get them working
     


Original Post:

This post is partly a rant, partly a cry for help, partly an inquiry about any tips and tricks that could improve our day-to-day work.

My company adopted a new, modern JS stack; all pretty standard:

  • React

  • Babel ES6 + JSX

  • Redux

  • Mocha

  • Webpack

  • React Router

  • lodash

  • along with their 1262 transitive dependencies (I'm not kidding here!)

As a result, my daily productivity plummeted. Honestly, I feel like x86 Assembly must have a friendlier set of developer tools.

A non-exhaustive list of issues:

  • We used to laugh at Java guys with their sluggish Maven, at C++ compilation times.
    Nowadays it takes Webpack longer to build a todo app than for a complex Java project to download all dependencies, clean project, compile all files, run all tests, create a package and upload it to repository. npm is even worse

  • So...Webpack's hot reload, the mythical feature getting universal praise.
    Apart from the fact that "hot" means "after 5 or more seconds", it comes with the worst of both worlds - sometimes it does work, sometimes it just doesn't; with no indication whatsoever whether it succeeded or not. So you spend 2 minutes figuring out why your fix didn't work, only to find out it hasn't been deployed in the first place. A colleague of mine stated: "I just don't trust hot reload. I always reload the entire page"

  • Webpack or Babel or something comes with a special, depression inducing feature: if you make a syntax error in your file, instead of displaying a clear error, it loads some random cached build from half a day ago

  • my employer can definitely find a better use for its money than me tracing build bugs of some deeply nested transitive 3rd party library that whimsically has changed it's AMD/UMD exports

  • so...source maps. I can see my original sources in Developer Tools! Usually, that is. So, let's try to add a debug breakpoint to my code!
    If you are lucky, it will stay on the line you clicked. Other times, it jumps a few lines up or down. Occasionally, it jumps to the bottom of the file. When developer tools sense that you haven't been frustrated for full 10 minutes, the breakpoint jumps to a 200kLoC app.js file, which freezes the entire Dev Tools and Chrome with it. Now, don't close Chrome at this point! If you do so, the next time you open Dev Tools, it will try to reopen the last session and freeze again. Just wait patiently for 2 minutes till Dev Tools finishes parsing the file, grab a cup of coffee. Actually, an antidepressant might be a better choice.

  • you won a lottery and Dev Tools really did stop at your breakpoint. Congrats,. Now, let's inspect some expression. Uncaught ReferenceError: X is not defined. What? Well, Babel thinks the best way to compile this piece of code:
    moment.isDate(...)
    is to translate it to:
    _moment2.default.isDate(...)
    This is done to every ES6 import. And yes, that "2" in _moment2 can and will change to "3" or just disappear, at any point.

  • Let's write some tests. Hm...they failed, let's see what's happening. Well, Chrome executes your test before it loads your sourcemaps, so it totally ignores your breakpoints. Remember that 200kLoC app.js file? That's your best bet now.

  • One of the best features of Javascript was an interactive REPL console in the browser. But that's so 2014! With Webpack/Babel, it's just not possible to get a reference to a particular module from Dev Tools Console. By design. Because best practices, man!

  • On the other hand, we have the 8th Wonder of the World, a time-travelling debugger for Redux. Last time I productively used it was never.

 

Is this...normal?  

How do you cope it?  

How is this clusterfuck touted as a state of the art in Javascript?  

Did you also become alergic to the phrase

makes it easy to reason about

 

516 Upvotes

292 comments sorted by

View all comments

12

u/gaearon Nov 30 '16 edited Nov 30 '16

A colleague of mine stated: "I just don't trust hot reload. I always reload the entire page"

Then why are you using it? Why are you not removing it?

A large part of the problem is when people integrate technologies clearly labeled as experimental, and expect them to be stable. They're just not, and never were. Please don't integrate something if it doesn't work well for you.

We're trying to figure out a minimal setup with hidden dependencies in Create React App that only includes stable software that works together well. I recommend you to give it a try and maybe adopt some ideas (even if you can't use it directly).

Cheers!

-14

u/mikrosystheme [κ] Nov 30 '16

To be 100% honest, I think that a large part of the problem is when people like you, working for huge and evil companies, create and evangelize technologies that are not needed by most people, painting such a complex set of tools as something "normal". Leave us (those who don't enjoy your "modern" tools) alone, please.

14

u/thejameskyle Dec 01 '16

Hi, I'm one of the original members of the Babel team and I did much of the work to evangelize it in the community.

I wish there was a "huge and evil corporation" funding the work on Babel. It wouldn't have so many problems if that were the case.

When Sebastian and I started Babel we were not working for companies that were paying us to work on it. Even today: We both work at Facebook and are paid to work on other things (I work on Flow and Sebastian works on JS Tools).

I actually worked with Facebook almost 2 years ago to get them to adopt Babel. They didn't originally want to because they had their own tooling which we needed to convince them to deprecate.

In fact, I did this with tons of people. In my free time I met with:

  • Frameworks: React, Ember, Angular, Aurelia, & more

  • Companies: Facebook, Google, Mozilla, Netflix, PayPal, & more

  • Editors: Atom, WebStorm

Not to mention hundreds and hundreds of hours of answering community questions, writing docs & blog posts, giving talks, putting together a handbook (and getting it translated to like 15 languages).

Babel became successful because I (and others) sacrificed my own personal time for over a year in order to make it successful.

Just when things were reaching critical mass, the worst possible thing happened: We got tired.

It turns out when open source is entirely powered by people who are working it in their free time you risk completely draining them. Open source burnout is a very real thing and I can tell you it really fucking sucks.

If Babel had been sponsored development, it would (without a single doubt) be in a much better place today and complaints about it would be minimal.

3

u/marosurbanec Dec 01 '16

Don't get discouraged. You've done heroic work.

The problem is not Babel or Redux or Webpack in and of itself. It's only when you start compounding edge-cases and pitfalls of all the 8-10 technologies in a stack, then it gets frustrating.

If it was up to me, I'd not consider such a stack mature enough to bet a chunk of company's future on it. The decision was not mine to make, though.

1

u/mikrosystheme [κ] Dec 02 '16

Hallo James. You are all focused about the "funding" part. To me money are insignificant, and the root of the problem. I think that our society tends to produce too much, just for the sake of producing, just to be a self-sustainable industry.

When you speak about personal projects, it looks like you did something (sacrificing your free time) that deserves a "thank you" a-priori. I think that this is a fallacy. You did it because you wanted to do it. Like every fucking thing that everybody with a passion in something does every day of their life. Your personal project happened to be something that became popular. You gained internet points. Your career advanced. You get employed by a big company. I am happy for you if this is what you were after.

I have different priorities in my life. I don't want to work for a big company. I don't want to get rich. I want to have the luxury to answer "No. Fuck you." to a client if their product is something that my moral consider execrable. In my free time I met with people that have nothing to do with programming, and I like it that way.

About being tired, it is normal. You are not a code-producing-machine. Relax. Take a break. If that issue is not going to be fixed ASAP, nobody is going to die, and if they really need it to be fixed, they can fix it themselves. It is open-source in the end, isnt it?

4

u/thejameskyle Dec 02 '16

Yeah, no. You're just wrong

1

u/mikrosystheme [κ] Dec 02 '16

Interesting argument. Care to articulate? Maybe I can change my mind.

12

u/gaearon Nov 30 '16 edited Dec 01 '16

Most of the tools you're presumably blaming (Babel, Webpack, React Hot Loader, Redux, Redux DevTools) are actually maintained by volunteers in their free time. I’m sure their state could be better if even 5% of the people complaining about them took time to contribute, whether with bug fixes, triaging, writing documentation and blog articles, helping beginners in issues, creating examples, and funding (yes, some of these projects, like webpack, actually accept money donations).

If you're speaking about React, it doesn't need a single tool, you can just drop a <script> on the page and write ES5 code if that's your preference.

That said I appreciate your fine point and will consider sharing less of my work with the community.

5

u/turtlecopter Dec 01 '16

will consider sharing less of my work with the community.

Don't you dare! :)

1

u/marosurbanec Dec 01 '16

It's not the tools I am blaming.

It's the bloggers and hipsters, shouting from the roottops, that such a stack should be used is ready for production use.

0

u/mikrosystheme [κ] Dec 02 '16

You managed to totally miss the point, and used it to enforce your point of view on your twitter echo chamber, rallying your followers. Thanks for the (undeserved) visibility you gave me, by the way.

Just a couple of points (where my usage of "you" is just a placeholder. Nothing personal agains you, Dan. I think you are a nice and smart person that unfortunately is being eaten by an entity that you still don't understand completely):

  • I have zero interest in contributing to your projects. For free or not. With code or money (that I don't have).

  • I am not going to ever use React or anything with the Facebook brand on top of it. It is both a political and a technological statement.

  • If you don't get the political meaning, it is not my job to convince you. Everyone is free to have their ideas, but they should consider the implications that their job has besides the technological aspect.

  • When you are employed, your free time is your employer's property.

  • I think that open source is the degeneration of free software.

  • I think that NPM Inc is evil and bad for the ecosystem. Azer was one of the few that manage to do the right thing. Almost nobody understood what he wanted to say with his statement, and focused on the insignificant part of it.

  • The economy you are part of is completely fucked-up and not sustainable.

  • Do whatever you want. Share everything you do. Show the typeface you bought and the editor colors you find nice. Make videos. Speak at conferences. Whatever. I don't care.

That being said, thank you for taking the time to respond to my message, and keep doing what you enjoy the most. Everything else is trifting.

6

u/gaearon Dec 05 '16 edited Dec 05 '16

I don’t think pointing out factually incorrect information in your comment is “enforcing” my point of view but whatever. I’m glad that your comment gave me an opportunity to direct more funds to people creating great open source software, and help break out of the unsustainable cycle (that’s something we agree on!)

Thanks for sharing your outlook. Perhaps when I’m older I’ll get more cynical, see open source as a zero-sum game, and see people as cogs in the machine too self-absorbed to consider moral implications of their work.

Or maybe I’ll keep thinking open source software makes technology more accessible, like it did for me, and together we can overcome setbacks regardless of who funds that work.

Chat again in 20 years?

2

u/mikrosystheme [κ] Dec 05 '16

RemindMe! 20 years "Chat with Dan about OSS superstars, evil companies exploiting their brain and the political/ethical meaning of our public actions."

1

u/RemindMeBot Dec 05 '16

I will be messaging you on 2036-12-05 22:22:16 UTC to remind you of this link.

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.


FAQsCustomYour RemindersFeedbackCodeBrowser Extensions

4

u/TheLarkInn Dec 01 '16

Well I'll speak for webpack and say that I evangelize because communication and representing apparently very trollworthy users like yourself is still vital to the success of a project. So if you got great ideas, throw them on the table so we can make webpack better.

Otherwise, if your just looking to vent for the sake venting, continue, and we'll keep trying to make it better anyways <3. In the end bundling is vital to performance.

1

u/mikrosystheme [κ] Dec 02 '16

Hallo. Thanks for painting me as a "venting troll". About the rest of your message, I don't think that bundling is vital to performance in the majority of the cases. Instead, I think that it is something that just adds a huge amount of complexity, and that break the linearity of the "view source" that makes the web a very nice platform to study other people's code and learn from it. The modern pipeline is so complex that un untrained eye gets lost into implementation details that leaks to the surface.
Anyway, thanks for the offer, but I have no interest in contributing to your project. My personal projects are already piling up (since I think quicker than I can manage to create), and I prefer to spend my life trying to give a form to my ideas instead of working on other people's ones.

2

u/TheLarkInn Dec 03 '16

Well the offer is on the table :) good luck!

2

u/lucidlogik Dec 01 '16

It's entirely possible to build ReactJS the "vanilla" way (via http://jamesknelson.com/learn-raw-react-no-jsx-flux-es6-webpack/)

But I have found that taking the time to learn each individual technology, be it Webpack, Babel, or whatever, is far more beneficial than cloning some boilerplate, where some dev has already done the grunt work for himself, and then gave simply it to the community. I wanted to be the person making the boilerplate.

And I didn't arrive at this methodology by chance since, but entirely the opposite. Like lots of folks, I learn by mimicking. So what'd I do? Cloned the most-starred boilerplates with all the bells & whistles that A Good Application Needs To Have. Just npm install and npm start and just hope for the best right? Wrong. I ended up frustrated like the OP. Shit was happening under the hood that I didn't understand and that made me uncomfortable, which is code for my boss is going to chew my ass out if any of this break and I have no understanding why.

The solution? Stop. Learn everything tech individually, then string them together to create an application. Just my 2 cents, YMMV.

2

u/marosurbanec Dec 01 '16

Out of curiosity - how to you even learn Webpack?

All I have ever seen anyone doing is a trial-and-error process, copying vaguely related snippets from Github or Stackoverflow, praying that this will be the magic config that will solve the issue. Pure alchemy.

I have never met anyone who could claim "I truly do understand Webpack".

3

u/gaearon Dec 01 '16

I (mostly) understand Webpack. This is why I’m working on a tool that hides its configuration from you. I hope you’ll find it helpful.

Webpack team is also working on new improved documentation at https://webpack.js.org/. I hope you’ll find it helpful.

2

u/lucidlogik Dec 01 '16 edited Dec 01 '16

For instance, understanding the order of operations in your configs module.loaders. For a given test, each loader is processed from right to left, not left to right. The ability to extract certain file extensions, like CSS, so you don't have one monolithic bundle.js file, using the ExtractCssPlugin. And just little hints and tweaks like this. You can certainly "webpackify" a simple JS file (not an entire app) just to get the feel for the flow.

edit: Oh, and check out the webpack confit that ships with create react app. It's heavily commented and I learned a ton from this as well: https://github.com/facebookincubator/create-react-app/blob/master/packages/react-scripts/config/webpack.config.dev.js