Front-End Ops

Advertisement

When a team builds a complex application, there is often a common breakdown of roles. Specifically on the back end, there are database engineers, application engineers and operations engineers, or something close to this. In recent years, more and more application logic is being deferred to the client side. For some reason, though, operations folks aren’t going with it.

I recently wrote an article on “Deploying JavaScript Applications.” It was largely well received, and I was happy with the content, but one negative comment stuck out to me. I probably didn’t have the reaction that the commenter was intending, but it pointed out something to me nonetheless.

“With all due respect, may I ask if you actually enjoy your job? I am a dev, and I do enjoy using tech to do stuff to a point. If your role is to squeeze every last second of performance out of your app, then yea, all this stuff must be cool. BUT if you are a coder doing something else and then come back to all of this as well, then wow, I don’t know how you haven’t gone mad already. I’d be sick to the stomach if I had to do all of this, in addition to my usual work.”

See, I had written my article with a few too many assumptions. I understood ahead of time that a few of my solutions weren’t globally applicable, and that many people wouldn’t have the time or energy to implement them. What I didn’t fully grasp was how different the role in that article is from the picture that people have of a front-end developer in their head. Up to this point, a front-end developer had just the few operations duties lumped into their role, and even then, many people chose to skip those steps (that’s why Steve Souders is constantly yelling at you to make your pages faster).

I think things are about to shift, and I’d (humbly) like to help guide that shift, because I think it’ll be great for the Web.

The Front-End Operations Engineer

A front-end operations engineer is not a title you’ve likely come across, but hopefully one that you will. Such a person would need to be an expert at serving and hosting front-end resources. They’d need to be pros at Grunt (or something similar) and have strong opinions about modules. They would find the best ways to piece together the parts of a Web application, and they’d be pros at versioning, caching and deployment.

A front-end operations engineer would own external performance. They would be critical of new HTTP requests, and they would constantly be measuring file size and page-load time. They wouldn’t necessarily always worry about the number of times that a loop can run in a second — that’s still an application engineer’s job. They own everything past the functionality. They are the bridge between an application’s intent and an application’s reality.

A front-end operations engineer would be very friendly with the quality assurance team, and they would make sure that “performance” is a test that comes up green. They’d monitor client-side errors and get alerts when things go wrong. They’d make sure that migrations to new versions of the application go smoothly, and they’d keep all external and internal dependencies up to date, secure and stable. They are the gatekeepers of the application.

Why?

We have reached a point where there is enough work to be done in the operations space that it often no longer serves us to have an application engineer do both jobs. When the application’s features are someone’s priorities, and that person has a full plate, they will typically deprioritize the critical steps in delivering their application most successfully to the end users.

Not every company or team can afford this person, but even if someone puts on the “front-end operations” hat for one day a week and prioritizes their work accordingly, users win. It doesn’t matter how many features you have or how sexy your features are if they aren’t delivered to the user quickly, with ease, and then heavily monitored. Front-end operations engineers are the enablers of long-term progress.

Builds And Deployment

If you were to ask most back-end engineers which person on their team has traditionally worried about builds and deployment, I’m sure you’d get a mixed bag. However, a very sizeable chunk of engineers would tell you that they have build engineers or operations engineers who handle these things. In that world, this often entails generating an RPM file, spinning up EC2 instances, running things through continuous integration tools, and switching load balancers over to new machines. Not all of this will necessarily go away for a front-end operations engineer, but there will be new tools as well.

A front-end operations engineer will be a master of the build tool chain. They’ll help run and set up the continuous integration (or similar) server but, more specifically, they’ll set up the testing instances that their application runs on and then, eventually, the deployment instances. They’ll integrate Git post-commit hooks into the application and run the tests (either in Node.js and PhantomJS or against something like Sauce Labs, Testling or BrowserStack) before anything gets merged into the master. They’ll need to make sure that those servers can take the raw code and, with a few commands, build up the resulting application.

This is where many people use Grunt these days. With a quick grunt build, these machines could be serving the built version of an application in order to enable proper testing environments. The front-end operations engineer would be in charge of much that’s behind that command as well. grunt build could call out to RequireJS’ r.js build tool, or a Browserify process, or it could simply minify and concatenate a list of files in order. It would also do similar things to the CSS (or your favorite preprocessed CSS dialect), in addition to crushing images, building sprites and reducing requests in any other way necessary or possible.

Front-end operations engineers would make sure that all of this stuff works on people’s local machines. A quick grunt test should be able to build everything locally, serve it and test it (likely with some WebDriver API-compatible server). They’d make sure that team members have the power to push their applications into the continuous integration environment and test them there. And they’d remove single points of failure from deployment (GitHub being down during launch wouldn’t scare them).

They’d facilitate internal deployments of feature branches and future release branches. They’d make sure that the quality assurance team has an easy time of testing anything and that the managers have an easy time of demoing things that aren’t ready.

They’d help build multiple versions of an application to best suit each of their core sets of users. This could mean builds for mobile or for old versions of Internet Explorer, but all of it should be relatively transparent to those who are programming against those feature, browser or device tests.

They’d facilitate the process of taking a release, building it, uploading it to a static edge-cached content delivery network, and flipping the switch to make it live. And they’d have a documented and fast roll-back mechanism in place.

Perhaps most importantly, they’d automate everything.

front-end ops start image_mini
(Image credits: Rube Goldberg)

Tracking Speed

The metric by which a front-end operations engineer would be judged is speed: the speed of the application, the speed of the tests, of the builds and deployment, and the speed at which other teammates understand the operational process.

A front-end operations engineer would live in a dashboard that feeds them data. Data is king when it comes to speed. This dashboard would integrate as much of it as possible. Most importantly, it would constantly be running the team’s app in multiple browsers and tracking all important metrics of speed. This space currently doesn’t have a ton of options, so they’d likely set up a private cloud of WebPageTest instances. They’d put them in multiple zones around the world and just run them non-stop.

They’d run against production servers and new commits and pull requests and anything they can get their hands on. At any given point, they’d be able to tell when, where, and what the surrounding circumstances were behind a slow-down. A decrease in speed would be directly correlated to some change, whether a new server, a diff of code, a dependency or third-party outage, or something similar.

They’d have a chart that graphs the number of HTTP requests on load. They’d also have a chart that tells them the Gzip’ed and minified payload of JavaScript, CSS and images that are delivered on load. And they’d also go crazy and have the unGzip’ed payload of JavaScript so that they can measure the effect of code parsing, because they know how important it can be on mobile. They’d instrument tools like mod_pagespeed and nginx_pagespeed to catch any mistakes that fall through the cracks.

They’d be masters of the latest development and measurement tools. They’d read flame graphs and heap snapshots of their apps from their development tools (in each browser that has them). They’d measure frames per second on scrolling and animations, prevent layout thrashing, build memory profiles, and keep a constant eye on compositing, rendering and the overall visual performance of the application. They’d do all of this for desktop and mobile devices, and they’d track trends in all of these areas.

They’d religiously parallelize tasks. They’d track the application via waterfalls and .har data to make sure that all serial operations are necessary or intentional.

They’d chart the average run time of the tests, builds and deploys. And they’d fight to keep them low. They’d chart their external dependencies in size and speed. They may not have control over slow API requests, but they’d want to be able to point to the reasons why their numbers are increasing.

They’d set an alarm if any of these numbers rose above an acceptable limit.

Monitoring Errors And Logs

Managing logging is a critical job of a normal operations engineer. The data that is generated from running an application is vital to understanding where things go wrong in the real world. A front-end operations engineer would also instrument tools and code that allow the same level of introspection on the client side.

This would often manifest itself as an analytics tool. Application engineers would be encouraged to log important events and any errors at certain levels to a logging service. These would be appropriately filtered and batched on the client and sent back as events to an internal or external analytics-style provider. The engineer would have enough information to identify the circumstances, such as browser name and version, application deployment version, screen size and perhaps a bit of other data. (Though they’d want to avoid storing personally identifiable information here.)

Logging stack traces can be very helpful in browsers that support them. You can integrate third-party services that do this for you.

The front-end operations engineer would encourage a very small tolerance for errors. Any error that happened would be investigated and either fixed or logged differently. With the data that comes back, they should be able to visualize groups of errors by browser or by state information from the application. A threshold of errors would be allowed to occur, and when that is passed, engineers would be notified. Severities would be assigned, and people would be responsible for getting patches out or rolling back as necessary (with quick patches being heavily favored to roll backs).

Much like today’s operations people focus on the security of the systems they manage, a front-end operations engineer would have probes for XSS vulnerabilities and would constantly be looking for holes in the app (along with the quality assurance team).

A front-end operations engineer would have an up-to-date picture of the state of the application in production. This is challenging in the front-end world, because your application doesn’t run on your machines — but that makes it even more necessary.

Keeping Things Fresh and Stable

My favorite thing that good operations people who I’ve worked with in the past were really good at was keeping things up to date. For some applications, stability and security are so deeply necessary that caution is the larger priority; but in most cases, failure to keep dependencies and environments up to date is what causes applications to get stale over time. We’ve all worked on a project that’s four years old where all of the tools are very old versions of the ones we know, and getting good performance out of it is impossible.

A front-end operations engineer would be effective at keeping dependencies up to date and at removing cruft in systems. When the next version of jQuery is released, they’d use their skills to switch out the dependency in the application to work with the new version and then test it to validate the change. They’d keep Grunt up to date (and Node.js along with it). When WebP becomes viable, they’d automate moving the application’s images over to that format.

They’d work closely with the more architecture-oriented application engineers to make sure that the entire system still feels viable and is not lagging behind in any one area. They would keep on top of this stuff as often as possible. Updating a dependency here and there as you build is far easier than having a big “Update Everything” day. It encourages application developers to loosely couple dependencies and to build good, consistent interfaces for their own modules.

A front-end operations engineer makes it viable and fun to work on a project long after it’s new.

The Future

I’m sure plenty of commenters will tell me that these tasks have been going on for years, and plenty will tell me that they should be the concern of all developers on a team. I would agree with both statements. I am not introducing new concepts; I’m compiling tasks we’ve all been doing for years and giving them a name. I think this will help us build better tools and document better processes in the future.

The addition of this role to a team doesn’t absolve the other members of performance responsibilities. It’s just that right now, front-end operations are no one’s explicit priority on most of the teams that I’ve encountered, and because of that, they often get skipped in crunch time. I think there’s enough to be done, especially in the configuration and monitoring of these tools, outside of the normal job of a front-end engineer, to justify this role.

Most importantly, regardless of whether a new job comes from these tasks, or whether we solve the problem in a different way, we do all need to be conscious of the importance of solving these problems in some way. You simply can’t ignore them and still achieve reliable, robust, high-experience applications. Addressing these concerns is critical to the stability and longevity of our applications and to the happiness of programmers and users.

If we build with that in mind, it helps the Web win, and we all want the Web to win.

(al) (il) (ea)

↑ Back to top

Alex Sexton is an engineer at Stripe, working out of Austin, TX. He spends time contributing to open-source as a member of the jQuery and Modernizr teams. He's passionate about large client-side applications, internationalization, Texas JavaScript, and whiskey. He tweets as @SlexAxton.

  1. 1

    I have very rarely had to deal with “front-end” performance issues. The issues have always been with the “back-end” getting the data. Occasionally I have implemented “front-end” solutions to “back-end” performance issues, such as pre-fetching the most likely next set of data within a page as a background requests etc.

    -7
    • 2

      There is lots of work to be done on the backend, especially in large, data-intense applications, but the data shows that the majority of load time for the average website is after the html is already loaded (to the tune of ~80%). So that means that a 2x speed up in the front-end will result in a much faster page load than a 2x speed up on the back end. So it would be helpful for someone to spend at least as much time worrying about front-end performance as they do about back-end performance.

      12
  2. 3

    I’m with you guy, there’s always going to be more to do as the industries surrounding software are becoming increasingly complex to wire and keep wired, we are all doing more, understanding where the lines are drawn and blurred is essential…below are just a handful of front-end roles born out of the last few years….

    front end lead
    front end developer
    front end designer
    front end engineer
    front end user interface developer
    front end user interface designer
    front end operations engineer*
    front end software engineer
    front end quality assurance tester

    the front-end tide is well in motion…

    2
  3. 4

    This all sounds good, but how does this get into the budget?

    4
    • 5

      This is not a very hard thing to get into the budget with the appropriate data. Measure your application performance and quality with the available open source tools, and show your team how well or poorly it does. Then put that info against all the articles that exist to say that performance user experience is directly linked to revenue. Amazon found that a 100ms slow down resulted in a reduction of 1% (sooo big) of *total revenue*. These findings have been corroborated many times over. I think anyone determined enough to sell this ideology to their company could have at least some success in getting these types tasks into their backlogs (and therefore, the budget).

      4
  4. 6

    Paw DOTSEO Hellegaard

    June 11, 2013 3:28 pm

    Great post, im a front end engineer… with a SEO brain ;-)

    0
  5. 7

    I work for a Fortune 500 company and I’m probably the closest anyone in our team comes to a front-end ops. Our development team consists of strong back-end java developers who tolerate the front-end. I’m just the opposite. It is beyond tough, working against, time, budget and team experience set on back-end performance.

    I nearly threw myself out the window one team meeting when I brought up huge performance savings by simple reductions in HTTP requests. Their answer was that we have ample server power to handle it and it wasn’t on their radar at this time. WHAAA!!!?? You would rather throw $$$$ at this for hardware then to simply fix issues like this?

    I do what I can, when I can, but working in a system like this is going to be the early death of me.

    9
  6. 12

    This is great stuff. My question though is how does one even learn this and apply it practically? I’ve been trying to cross the boundary from being a front end developer to a front end engineer and it’s TOUGH. I have made many trivial apps on my own, but obviously they don’t get enough traffic to actually get me into the realms of performance optimization and the menagerie of tools/methods you discuss.

    And of course, this brings about a very common conundrum. If you don’t have experience in this area, you can’t get a job. And you can’t get real experience in this area until you have a job in it.

    For all the experienced app developers, how did you get to what most would consider advanced front end engineering?

    3
    • 13

      Lawrence,

      No doubt there will be plenty of ways to reach your goal. Although I really can’t claim to be any type of certified “front-end engineer”, I can at least share my experiences with you.

      The bulk of my experience and “expertise” comes from a continual search for a “better way”. Take what you learn and apply it in as many ways as you can. Share with colleagues, discuss online or build “trivial apps” like you are doing. I think it’s a rare few who actually get their experience/knowledge through class time.

      If you have a job already and no one has this position, make it yours. That’s what I have done. Expertise is in the eye of the beholder. I sure as hell don’t have the experience Alex has in this area, but because I constantly try to improve in this area, I’m the “in-house” expert at my company. People now seek me out for front-end advice.

      As for a creating more experience. Keep doing what you are doing. Why not look for some free load/stress testing service or open source application? Break your own work, tweak for performance, then try and break it again. Not only will it be an invaluable learning experience, you will have something to provide potential employers as validation of skills.

      3
  7. 14

    Alex,

    This is a great futuristic post. I think our team is on the cusp of needing someone to wear the operations hat and take ownership in that department. As a team we’ve sort of distributed the responsibilities that you’ve listed.

    good post.

    sam

    1
  8. 15

    Brilliant article! So weird, because I was consulting with a client only today about the use of Stripe, and whether it might come to Germany anytime soon.

    Back to the article, though. I’m an “all-round” sort of guy. I like work alone, so I usually tackle web stuff by myself, rather than in a team, whether it’s back-end or front-end development. This is enough to drive to me mad by itself, so I can understand the frustrations with trying to achieve perfect or near perfect front-end performance. Not putting enough effort in PageSpeed almost comes with a death threat these days, especially now Google has declared it a factor in ranking/seo.

    That being said, I was rather happy when I received a high 90′s in the PageSpeed score when I designed http://airwalk-design.com, so I suppose “driving yourself mad” about it is worthwhile, at least in my books.

    Again, great article, thanks.

    2
  9. 16

    Christian Johannessen

    June 11, 2013 9:27 pm

    Nice article. I sincerely hope a role like this becomes commonplace. If you happen to come across one of these mythical beings in the meantime, please send one our way…

    2
  10. 17

    I am a “traditional” frontend developer that comes from a time where all we had to worry about was IE6 and pixels that were a little bit off.

    In the last few years frontend work got very complex and basically branched out into different roles, as listed in Kyle Winterton’s comment above.

    Now is the time where the “sheep get divided from the goats” and if we want to continue succeeding as frontend developers we need to step up our game.

    The main issue I see is that many frontend devs (including myself) lack proper programming skills. We know how to push pixels around but when it comes to develop e.g. in a test-driven way we go “ummmmm, no idea”.

    Fancy and no doubt useful roles such as frontend-op only make sense if the quality of the code we do in the frontend increases :)

    3
    • 18

      This is exactly what I was addressing in my comment/question. Crossing over from the “traditional” front end developer to becoming a front end engineer seems to be a rather steep curve.

      0
  11. 19

    You should have mentioned the new window.performance object!

    0
  12. 20

    Peter McLachlan

    June 12, 2013 2:16 am

    Alex, I completely agree that there is a role here that isn’t filled in any but the most technically focused organizations. Even technology companies shipping primarily web products are inattentive to the level of shipping detail you discuss here — let alone owners of more typical web properties such as corporate websites or eCommerce sites.

    Most organizations with a web presence are stretched on technology resources as technology is viewed as a cost center. Many of these organizations may not even have in-house expertise but depend on agencies or contractors for their web team in which case these kinds of optimizations are going to be the first thing to get the axe when the project starts to run up to it’s budget, that is if they were even included in the first place!

    For these organizations I see more of a move more towards third party SaaS solutions for performance tracking & optimization, logging & monitoring, deployment optimization etc. These services will never beat a fulltime expert on staff, but that’s a luxury not many organizations have.

    1
  13. 21

    For anyone that is intrigued by this topic, I HIGHLY recommend that you start taking a look at the Continuous Delivery and DevOps movement that is occurring in industry these days. A lot of the details mentioned here are explored more deeply in those topics.

    I found that this article is a front end developer’s version of CAMS (Culture, Automation, Measurement, Sharing): http://www.opscode.com/blog/2010/07/16/what-devops-means-to-me/

    0
  14. 22

    Quite lengthy article but I would like to appreciate your work for taking time and explaining about the duties of the front-end operations engineer !! Nice post !!

    0
  15. 23

    Sounds like you’re combining elements from multiple roles: Performance Engineer, Software Test Engineer & Operations Engineer, Tools Engineer. How much focus needs to be front-end vs back end depends a lot on your stack and application type. There is something to be said for having someone focus on these issues exclusively, but you could accomplish similar things by prioritizing the right performance metrics. The reasons to have a specific position is just to mitigate priority conflicts and parallelize effort. Engineers are often bad at picking between feature work and optimization work when the business is breathing down their neck.

    0
  16. 24

    This makes me happy.

    In this model do you feel that larger projects will tend to devolve into a waterfall Dev vs Ops model where the front-end engineers and the front-end ops have competing priorities and the FEOps ™ becomes blocker/nag to the rest?

    0
  17. 25

    I ♥ being a frontend dev. Business in the back, party in front. :)

    2
  18. 26

    Great article myself Frontend Dev I Feel this is missing in our proses as we are small agency here time/budget is a factor and i am there only one working on front end side of things. so can get on top of me at the moment.

    Planning is key for me and i think allowing just one day a week to look a performance would be great help would be longer build time, Just a case of managing the client timescale expectations and budget but in the long run it ends in a better result.

    0
  19. 27

    I presented on this topic at Velocity Conf 2013 in Santa Clara and the work we did on CNN.com
    Slides: https://speakerdeck.com/wesleyhales/a-baseline-for-web-performance-with-phantomjs
    The project/dashboard born from this: http://wesleyhales.com/wesleyhales.com/ (click on a point within the first chart to see commit details)
    github: https://github.com/fluxui

    Still a ton of work (and polishing) to do :)

    0
  20. 28

    It’s a great post where any one can get the better idea to build a great web design.

    0
  21. 29

    I can only hope that a few Project Managers and Team leaders read this article and start to ‘get it’. Front-end matters, just as much as back-end. Allocate the appropriate resources, get your teams balanced and give people the time they need to do the best job they can.

    0
  22. 30

    Learning and doing all this stuff pays off only when working on a large project with a huge userbase. Every good front end developer should know some basic performance practices – minifying, concatenating, gzip, sprites, asset preloading/lazy loading etc. Those combined with writing clean html, css and js should be more than enough to create great user experience.

    I just don’t see space nor need for this kind of perfectionism in any project/startup with a reasonable budget. Successful (and huge) funded projects who can afford top developers are ofcourse a different story.

    In my humble opinion :-)

    0

Leave a Comment

Yay! You've decided to leave a comment. That's fantastic! Please keep in mind that comments are moderated and rel="nofollow" is in use. So, please do not use a spammy keyword or a domain as your name, or else it will be deleted. Let's have a personal and meaningful conversation instead. Thanks for dropping by!

↑ Back to top