opensource.google.com

Menu
Showing posts with label JavaScript. Show all posts
Showing posts with label JavaScript. Show all posts

Wombat Dressing Room, an npm publication proxy on GCP

Friday, January 10, 2020

We're excited to announce that we're open sourcing the service we use on the Google Cloud Client Libraries team for handling npm publications, it's called Wombat Dressing Room. Wombat Dressing Room provides features that help npm work better with automation, while maintaining good security practices.

A tradeoff is often made for automation

npm has top notch security features: CIDR-range restricted tokens, publication notifications, and two-factor authentication, to name a few. Of these, a feature critical to protecting publications is two-factor authentication (2FA).

2FA requires that you provide two pieces of information when accessing a protected resource: "something you know" (for instance, a password); and "something you have" (for instance, a code from an authenticator app). With 2FA, if your password is exposed, an attacker still can't publish a malicious package (unless they also steal the "something you have".)

On my team, a small number of developers manage over 75 Node.js libraries. We see automation as key to making this possible: we've written tools that automate releases, validate license headers, ensure contributors have signed CLAs; we adhere to the philosophy, automate all the things!

It's difficult to automate the step of entering a code off a cellphone. As a result, folks often opt to turn off 2FA in their automation.

What if you could have both automation and the added security of 2FA? This is why we built the Wombat Dressing Room.

A different approach to authentication

With Wombat Dressing Room, rather than an individual configuring two factor authentication in an authenticator app, 2FA is managed by a shared proxy server. Publications are then directed at the Wombat Dressing Room proxy, which provides the following security features:

Per-package publication tokens.

Wombat Dressing Room can generate authentication tokens tied to repositories on GitHub. These tokens are tied to a single GitHub repository, which the user generating the token must have push permissions for.

If a per-package publication token is leaked, an attacker can only hijack the single package that the token is associated with.

Limited lifetime tokens

Wombat Dressing Room can also generate access tokens that have a 24 hour lifespan. In this model, a leaked token is only vulnerable until the 24 hour lifespan is hit.

GitHub Releases as 2FA

In this authentication model, a package can only be published to npm if a GitHub release with a corresponding tag is found on GitHub.

This introduces a true "second factor", as users must prove they have access to both Wombat Dressing Room and the repository on GitHub.

Getting started with Wombat Dressing Room

We've been using Wombat Dressing Room to manage Google Cloud client libraries for over a year now in our fully automated library release process. As of today, the source is available for everyone on GitHub under an Apache 2.0 license.

Wombat Dressing Room runs on Google App Engine, and instructions on getting it up and running can be found in its README.md.

It's my hope that this will help other folks in the community, simplify and automate their release process, while minimizing the attack surface of their libraries.
By Benjamin Coe, works on Node.js client libraries for the Google Cloud Platform, and was the third engineer at npm, Inc.

OpenCensus Web: Unlocking Full End-to-End Observability for Your Entire Stack

Friday, August 9, 2019

OpenCensus Web is a tool to trace and monitor the user-perceived performance of your web pages. It can help determine whether or not your web pages are experiencing performance issues that you might otherwise not know how to diagnose.

Web application owners want to monitor the operational health of their applications so that they can better understand actual user performance; however, capturing relevant telemetry from your web applications is often very difficult. Today, we are introducing OpenCensus Web (OC Web) to make instrumenting and exporting metrics and distributed traces from web applications simple and automatic.

Background

Image for: Background
The OpenCensus project provides a set of language-specific instrumentation libraries that collect traces and metrics from applications and export them to tracing and monitoring backends like Prometheus, Zipkin, Jaeger, Stackdriver, and others.

The OpenCensus Web library is an implementation of OpenCensus that focuses on frontend web application code that executes in the browser. OC Web instruments web pages and collects user-side performance data, including latency and distributed traces, which gives developers the information to diagnose frontend issues and monitor overall application health.

Overshadowing the work on OC Web, the wider OpenCensus family of projects is merging with OpenTracing into OpenTelemetry. OpenCensus Web’s functionality will be migrated into OpenTelemetry JS once this project is ready, although OC Web will continue working as an alpha release in the meantime.

Architecture

Image for: Architecture
OC Web interacts with three application components:
  • Frontend web server: renders the initial HTML to the browser including the OC Web library code and configuration. This would typically be instrumented with an OpenCensus server-side library (Go, Java, etc.). We also suggest that you create an endpoint in the server that receives HTTP/JSON traces and proxies to the OpenCensus Agent.
  • Browser JS: the OC Web library code that runs in the browser. This measures user interactions and collects browser data and writes them to the OpenCensus Agent as spans via HTTP/JSON.
  • OpenCensus Agent: receives traces from the frontend web server proxy endpoint or directly from the browser JS, and exports them to a trace backend (e.g. Stackdriver, Zipkin).
OC Web requires the OpenCensus Agent, which will proxy and re-export telemetry to your backend of choice. For more details see the documentation.


Features

Image for: Features

Initial page load tracing

You can use OC Web to capture traces of initial page loads, which will even capture events that take place before the OC Web library was loaded by the browser! Initial page load traces show you which resources may be causing poor website performance, and contain data that you can’t typically capture from a distributed tracing system.

To measure the time of the overall initial page load interaction, OC Web waits until after the document load event and generates spans from the initial load performance timings via the browser's Navigation Timing and Resource Timing APIs. Below is a sample trace from OC Web that has been exported to Zipkin and captured from the initial load example app. Notice that there is an overall ‘nav./’ span for the user navigation experience until the browser load event fires.

This example also includes ‘/’ spans for the client and server side measurements of the initial HTML load. These spans are connected by the server sending back a ‘window.traceparent’ variable in the W3C Trace Context format, which is necessary because the browser does not send a trace context header for the initial page load. The server side spans also indicate how much time was spent parsing and rendering the template:

Notice the long js task span in the previous image, which indicates a CPU-bound JavaScript event loop that took 80.095ms, as measured by the Long Tasks browser API.

Span annotations for DOM and network events

Spans captured by OC Web also include detailed annotations for DOM events like `domInteractive` and `first-paint`, as well as network events like domainLookupStart and secureConnectionStart. Here is a similar trace exported to Stackdriver Trace with the annotations expanded:


User Interactions

For single page applications there are often subsequent interactions after the initial load (e.g. user clicks a button or navigates to a different section of the page). Measuring end-user interactions within a browser application adds useful data for your application:
  • Ability to relate an initial page render with subsequent on-page interactions
  • Visibility into slowness as perceived by the end user, for example, an unresponsive page after clicking
Currently, OC Web tracks clicks and route transitions by monkey-patching the Angular Zone.js library. OC Web tracks the subsequent synchronous and asynchronous tasks (e.g. setTimeouts, XHRs, etc.) caused by the interaction even if there are several concurrent interactions.

Automatic tracing for click events

All browser click events are traced as long as the click is done in a DOM element (e.g. button) and the clicked element is not disabled. When the user clicks the element, a new Zone is created to measure this interaction and determine the total time.

To name this root span, we provide developers with the option of adding the attribute data-ocweb-id to elements and give a custom name to the interaction. For the next example, the resulting name will be ‘Save edit user info’:
<button type="submit" data-ocweb-id="Save edit user info">       Save changes </button>
This helps you to identify the traces related to a specific element. Also, this may avoid ambiguity when there are similar interaction. If you don’t add this attribute, OC Web will use the DOM element ID, the tag name plus the event involved in the interaction. For example, clicking this button:
<button id="save_changes"> Save changes </button>
will generate a span named : “button#save_changes click”.

Automatic tracing for route transitions

OC Web traces route transitions between the different sections of your page by monkey-patching the History API. OC Web will name these interactions with the pattern ‘Navigation /path/to/page’. The following screenshot of a trace exported to Stackdriver from the user interaction example shows a Navigation trace which includes several network calls before the route transition is complete:

Creating your own custom spans

OC Web allows you to instrument your web application with custom spans for tasks or code involved in a user interaction. Here is a code snippet that shows how to do this:

import { tracing } from '@opencensus/web-instrumentation-zone';

function handleClick() {

  // Start child span of the current root span on the current interaction.
  // This must run in in code that the button is running.
  const childSpan = tracing.tracer.startChildSpan({
    name: 'name of your child span'
  });
  // Do some operation...
  // Finish the child span at the end of it's operation
  childSpan.end();
}

See the OC Web documentation for more details.

Automatic spans for HTTP requests and Browser performance data

OC Web automatically intercepts and generates spans for HTTP requests generated by user interactions. Additionally, OC Web attaches Trace Context Headers to each intercepted HTTP request, using the W3C Trace Context format. This is only done for same-origin requests or requests that match a provided regex.

If your servers are also instrumented with OpenCensus, these requests will continue to be traced throughout your backend services! This lets you know if the issues are related to either the front-end or the server-side.

OC Web also includes Performance API data to make annotations like domainLookupStart and responseEnd and generates spans for any CORS preflight requests.

The next screenshot shows a trace exported to Stackdriver as result of the user interaction example. There, you can see the several network calls with the automatic generated spans (e.g. ‘Sent./sleep’) with annotations, the server-side spans (e.g. ‘/sleep’ and ‘ocweb.handlerequest’) and CORS Preflight related spans:

Relate user interactions back to the initial page load tracing

OC Web attaches the initial page load trace id to the user interactions as an attribute and a span link. This enables you to do a trace search by attribute to find the initial load trace and its interactions traces via a single attribute query as well as letting you understand the whole navigation of a user through the application for a given page load.

The next screenshot shows a search by initial_load_trace_id attribute containing all user interaction traces after the initial page loaded:


Making it Real

Image for: Making it Real
With OC Web and a few lines of instrumentation, you can now export distributed traces from your web application. Start exploring the initial load and user interaction examples and you're welcome to poke around the source code and send us feedback via either Gitter or contributing with Pull Requests!

By Cristian González – OpenCensus Team – Software Engineering intern at Google Summer 2019 and student of Computer and Systems Engineering at Universidad Nacional de Colombia.

Special thanks to Dave Raffensperger for being initial creator of OC Web and guiding me in the process to develop i
t.

Talk to Google at Node.js Interactive

Monday, October 2, 2017

We’re headed to Vancouver this week, with about 25 Googlers who are incredibly excited to attend Node.js Interactive. With a mix of folks working on Cloud, Chrome, and V8, we’re going to be giving demos and answering questions at the Google booth.

A few of us are also going to be giving talks. Here’s a list of the talks Googlers will be giving at the conference, ranging from serverless Slack bots to JavaScript performance tuning.

Wednesday, October 4th

Image for: Wednesday, October 4th
Keynote: Franzi Hinkelmann
9:40am - 10:00am, West Ballroom A
Franzi Hinkelmann, Software Engineer @ Google

Franzi is located in Munich, Germany where she works at Google on Chrome V8. Franzi, like James and Anna, is a member of the Node.js Core Technical Committee. She speaks across the globe on the topic of JavaScript virtual machines. She has a PhD in mathematics, but left academia to follow her true passion: writing code.

Franzi will discuss her perspective on Chrome V8 in Node.js, and what the Chrome V8 team is doing to continue to support Node.js. Want to know what the future of browser development looks like? This is a must-attend keynote.

Functionality Abuse: The Forgotten Class of Attacks
12:20pm - 12:50pm, West Ballroom A
Nwokedi Idika, Software Engineer @ Google

If you were given a magic wand that would remove all implementation flaws from your web application, would it be free of security problems? If it took you more five seconds to say “No!” (or if, worse, you said “Yes!”), then you’re the target audience for this talk. If you’re in the target audience, don’t fret, much of the security community is there with you. After this talk, attendees will understand why the answer to the aforementioned question is an emphatic “No!” and they will learn an approach to decrease their chance of failing to consider an important vector of attack for their current and future web applications.

High Performance JS in V8
5:20pm - 5:50pm, West Ballroom A
Peter Marshall, Software Engineer @ Google

This year, V8 launched Ignition and Turbofan, the new compiler pipeline that handles all JavaScript code generation. Previously, achieving high performance in Node.js meant catering to the oddities of our now-deprecated Crankshaft compiler. This talk covers our new code generation architecture - what makes it special, a bit about how it works, and how to write high performance code for the new V8 pipeline.

Thursday, October 5th

Image for: Thursday, October 5th
New DevTools Features for JavaScript
11:40am - 12:10pm, West Meeting Room 122
Yang Guo, Software Engineer @ Google

Ever since v8-inspector was moved to V8's repository, we have been working on a number of new features for DevTools, usable for both Chrome and Node.js. The talk will demonstrate code coverage, type profiling, and give a deep dive into how evaluating a code snippet in DevTools console works in V8.

Understanding and Debugging Memory Leaks in Your Node.js Applications
12:20pm - 12:50pm, West Meeting Room 122
Ali Sheikh, Software Engineer @ Google

Memory leaks are hard. This talk with introduce developers to what memory leaks are, how they can exist in a garbage collected language, the available tooling that can help them understand and isolate memory leaks in their code. Specifically it will talk about heap snapshots, the new sampling heap profiler in V8, and other various other tools available in the ecosystem.

Workshop: Serverless Bots with Node.js
2:20pm - 4:10pm, West Meeting Room 117
Bret McGowen, Developer Advocate @ Google
Amir Shevat, DevRel @ Slack

This talk will show you how to build both voice and chat bots using serverless technologies. Amir Shevat, Head of Developer Relations at Slack, has overseen 17K+ bots deployed on the platform. He will present a maturing model, as best practices, for enterprise bots covering all sorts of use cases ranging for devops, HR, and marketing. Alan Ho from Google Cloud will then show you how to use various serverless technologies to build these bots. He’ll give you a demo of Slack and Google Assistant bots incorporating Google’s latest serverless technology including Edge (API Management), CloudFunctions (Serverless Compute), Cloud Datastore, and API.ai.

Modules Modules Modules
3:00pm - 3:30pm, West Meeting Room 120
Myles Borins, Developer Advocate @ Google

ES Modules and Common JS go together like old bay seasoning and vanilla ice cream. This talk will dig into the inconsistencies of the two patterns, and how the Node.js project is dealing with reconciling the problem. The talk will look at the history of modules in the JavaScript ecosystem and the subtle difference between them. It will also skim over how ECMA-262 is standardized by the TC39, and how ES Modules were developed.

Keynote: The case for Node.js
4:50pm - 5:05pm, West Ballroom A
Justin Beckwith, Product Manager @ Google

Node.js has had a transformational effect on the way we build software. However, convincing your organization to take a bet on Node.js can be difficult. My personal journey with Node.js has included convincing a few teams to take a bet on this technology, and this community. Let’s take a look at the case for Node.js we made at Google, and how you can make the case to bring it to your organization.


We can’t wait to see everyone and have some great conversations. Feel free to reach out to us on Twitter @googlecloud, or request an invite to the Google Cloud Slack community and join the #nodejs channel.

By Justin Beckwith, Languages and Runtimes Team

Angular, version 2: proprioception-reinforcement

Thursday, September 22, 2016

Originally posted on the Angular Blog

Today, at a special meetup at Google HQ, we announced the final release version of Angular 2, the full-platform successor to Angular 1.

What does "final" mean? Stability that's been validated across a wide range of use cases, and a framework that's been optimized for developer productivity, small payload size, and performance. With ahead-of-time compilation and built-in lazy-loading, we’ve made sure that you can deploy the fastest, smallest applications across the browser, desktop, and mobile environments. This release also represents huge improvements to developer productivity with the Angular CLI and styleguide.

Angular 1 first solved the problem of how to develop for an emerging web. Six years later, the challenges faced by today’s application developers, and the sophistication of the devices that applications must support, have both changed immensely. With this release, and its more capable versions of the Router, Forms, and other core APIs, today you can build amazing apps for any platform. If you prefer your own approach, Angular is also modular and flexible, so you can use your favorite third-party library or write your own.

From the beginning, we built Angular in collaboration with the open source development community. We are grateful to the large number of contributors who dedicated time to submitting pull requests, issues, and repro cases, who discussed and debated design decisions, and validated (and pushed back on) our RCs. We wish we could have brought every one of you in person to our meetup so you could celebrate this milestone with us tonight!


What’s next?

Angular is now ready for the world, and we’re excited for you to join the thousands of developers already building with Angular 2.  But what’s coming next for Angular?

A few of the things you can expect in the near future from the Angular team:

  • Bug fixes and non-breaking features for APIs marked as stable
  • More guides and live examples specific to your use cases
  • More work on animations
  • Angular Material 2
  • Moving WebWorkers out of experimental
  • More features and more languages for Angular Universal
  • Even more speed and payload size improvements

Semantic Versioning

We heard loud and clear that our RC labeling was confusing. To make it easy to manage dependencies on stable Angular releases, starting today with Angular 2.0.0, we will move to semantic versioning.  Angular versioning will then follow the MAJOR.MINOR.PATCH scheme as described by semver:

  1. the MAJOR version gets incremented when incompatible API changes are made to stable APIs,
  2. the MINOR version gets incremented when backwards-compatible functionality are added,
  3. the PATCH version gets incremented when backwards-compatible bug are fixed.

Moving Angular to semantic versioning ensures rapid access to the newest features for our component and tooling ecosystem, while preserving a consistent and reliable development environment for production applications that depend on stability between major releases, but still benefit from bug fixes and new APIs.

Contributors

Aaron Frost, Aaron (Ron) Tsui, Adam Bradley, Adil Mourahi, agpreynolds, Ajay Ambre, Alberto Santini, Alec Wiseman, Alejandro Caravaca Puchades, Alex Castillo, Alex Eagle, Alex Rickabaugh, Alex Wolfe, Alexander Bachmann, Alfonso Presa, Ali Johnson, Aliaksei Palkanau, Almero Steyn, Alyssa Nicoll, Alxandr, André Gil, Andreas Argelius, Andreas Wissel, Andrei Alecu, Andrei Tserakhau, Andrew, Andrii Nechytailov, Ansel Rosenberg, Anthony Zotti, Anton Moiseev, Artur Meyster, asukaleido, Aysegul Yonet, Aziz Abbas, Basarat Ali Syed, BeastCode, Ben Nadel, Bertrand Laporte, Blake La Pierre, Bo Guo, Bob Nystrom, Borys Semerenko, Bradley Heinz, Brandon Roberts, Brendan Wyse, Brian Clark, Brian Ford, Brian Hsu, dozingcat, Brian Yarger, Bryce Johnson, CJ Avilla, cjc343, Caitlin Potter, Cédric Exbrayat, Chirayu Krishnappa, Christian Weyer, Christoph Burgdorf, Christoph Guttandin, Christoph Hoeller, Christoffer Noring, Chuck Jazdzewski, Cindy, Ciro Nunes, Codebacca, Cody Lundquist, Cody-Nicholson, Cole R Lawrence, Constantin Gavrilete, Cory Bateman, Craig Doremus, crisbeto, Cuel, Cyril Balit, Cyrille Tuzi, Damien Cassan, Dan Grove, Dan Wahlin, Daniel Leib, Daniel Rasmuson, dapperAuteur, Daria Jung, David East, David Fuka, David Reher, David-Emmanuel Divernois, Davy Engone, Deborah Kurata, Derek Van Dyke, DevVersion, Dima Kuzmich, Dimitrios Loukadakis, Dmitriy Shekhovtsov, Dmitry Patsura, Dmitry Zamula, Dmytro Kulyk, Donald Spencer, Douglas Duteil, dozingcat, Drew Moore, Dylan Johnson, Edd Hannay, Edouard Coissy, eggers, elimach, Elliott Davis, Eric Jimenez, Eric Lee Carraway, Eric Martinez, Eric Mendes Dantas, Eric Tsang, Essam Al Joubori, Evan Martin, Fabian Raetz, Fahimnur Alam, Fatima Remtullah, Federico Caselli, Felipe Batista, Felix Itzenplitz, Felix Yan, Filip Bruun, Filipe Silva, Flavio Corpa, Florian Knop, Foxandxss, Gabe Johnson, Gabe Scholz, GabrielBico, Gautam krishna.R, Georgii Dolzhykov, Georgios Kalpakas, Gerd Jungbluth, Gerard Sans, Gion Kunz, Gonzalo Ruiz de Villa, Grégory Bataille, Günter Zöchbauer, Hank Duan, Hannah Howard, Hans Larsen, Harry Terkelsen, Harry Wolff, Henrique Limas, Henry Wong, Hiroto Fukui, Hongbo Miao, Huston Hedinger, Ian Riley, Idir Ouhab Meskine, Igor Minar, Ioannis Pinakoulakis, The Ionic Team, Isaac Park, Istvan Novak, Itay Radotzki, Ivan Gabriele, Ivey Padgett, Ivo Gabe de Wolff, J. Andrew Brassington, Jack Franklin, Jacob Eggers, Jacob MacDonald, Jacob Richman, Jake Garelick, James Blacklock, James Ward, Jason Choi, Jason Kurian, Jason Teplitz, Javier Ros, Jay Kan, Jay Phelps, Jay Traband, Jeff Cross, Jeff Whelpley, Jennifer Bland, jennyraj, Jeremy Attali, Jeremy Elbourn, Jeremy Wilken, Jerome Velociter, Jesper Rønn-Jensen, Jesse Palmer, Jesús Rodríguez, Jesús Rodríguez, Jimmy Gong, Joe Eames, Joel Brewer, John Arstingstall, John Jelinek IV, John Lindquist, John Papa, John-David Dalton, Jonathan Miles, Joost de Vries, Jorge Cruz, Josef Meier, Josh Brown, Josh Gerdes, Josh Kurz, Josh Olson, Josh Thomas, Joseph Perrott, Joshua Otis, Josu Guiterrez, Julian Motz, Julie Ralph, Jules Kremer, Justin DuJardin, Kai Ruhnau, Kapunahele Wong, Kara Erickson, Kathy Walrath, Keerti Parthasarathy, Kenneth Hahn, Kevin Huang, Kevin Kirsche, Kevin Merckx, Kevin Moore, Kevin Western, Konstantin Shcheglov, Kurt Hong, Levente Morva, laiso, Lina Lu, LongYinan, Lucas Mirelmann, Luka Pejovic, Lukas Ruebbelke, Marc Fisher, Marc Laval, Marcel Good, Marcy Sutton, Marcus Krahl, Marek Buko, Mark Ethan Trostler, Martin Gontovnikas, Martin Probst, Martin Staffa, Matan Lurey, Mathias Raacke, Matias Niemelä, Matt Follett, Matt Greenland, Matt Wheatley, Matteo Suppo, Matthew Hill, Matthew Schranz, Matthew Windwer, Max Sills, Maxim Salnikov, Melinda Sarnicki Bernardo, Michael Giambalvo, Michael Goderbauer, Michael Mrowetz, Michael-Rainabba Richardson, Michał Gołębiowski, Mikael Morlund, Mike Ryan, Minko Gechev, Miško Hevery, Mohamed Hegazy, Nan Schweiger, Naomi Black, Nathan Walker, The NativeScript Team, Nicholas Hydock, Nick Mann, Nick Raphael, Nick Van Dyck, Ning Xia, Olivier Chafik, Olivier Combe, Oto Dočkal, Pablo Villoslada Puigcerber, Pascal Precht, Patrice Chalin, Patrick Stapleton, Paul Gschwendtner, Pawel Kozlowski, Pengfei Yang, Pete Bacon Darwin, Pete Boere, Pete Mertz, Philip Harrison, Phillip Alexander, Phong Huynh, Polvista, Pouja, Pouria Alimirzaei, Prakal, Prayag Verma, Rado Kirov, Raul Jimenez, Razvan Moraru, Rene Weber, Rex Ye, Richard Harrington, Richard Kho, Richard Sentino, Rob Eisenberg, Rob Richardson, Rob Wormald, Robert Ferentz, Robert Messerle, Roberto Simonetti, Rodolfo Yabut, Sam Herrmann, Sam Julien, Sam Lin, Sam Rawlins, Sammy Jelin, Sander Elias, Scott Hatcher, Scott Hyndman, Scott Little, ScottSWu, Sebastian Hillig, Sebastian Müller, Sebastián Duque, Sekib Omazic, Shahar Talmi, Shai Reznik, Sharon DiOrio, Shannon Ayres, Shefali Sinha, Shlomi Assaf, Shuhei Kagawa, Sigmund Cherem, Simon Hürlimann (CyT), Simon Ramsay, Stacy Gay, Stephen Adams, Stephen Fluin, Steve Mao, Steve Schmitt, Suguru Inatomi, Tamas Csaba, Ted Sander, Tero Parviainen, Thierry Chatel, Thierry Templier, Thomas Burleson, Thomas Henley, Tim Blasi, Tim Ruffles, Timur Meyster, Tobias Bosch, Tony Childs, Tom Ingebretsen, Tom Schoener, Tommy Odom, Torgeir Helgevold, Travis Kaufman, Trotyl Yu, Tycho Grouwstra, The Typescript Team, Uli Köhler, Uri Shaked, Utsav Shah, Valter Júnior, Vamsi V, Vamsi Varikuti, Vanga Sasidhar, Veikko Karsikko, Victor Berchet, Victor Mejia, Victor Savkin, Vinci Rufus, Vijay Menon, Vikram Subramanian, Vivek Ghaisas, Vladislav Zarakovsky, Vojta Jina, Ward Bell, Wassim Chegham, Wenqian Guo, Wesley Cho, Will Ngo, William Johnson, William Welling, Wilson Mendes Neto, Wojciech Kwiatek, Yang Lin, Yegor Jbanov, Zach Bjornson, Zhicheng Wang, and many more...

With gratitude and appreciation, and anticipation to see what you'll build next, welcome to the next stage of Angular.

By Jules Kremer, Angular Team

From Google Summer of Code to Game of Thrones on the Back of a JavaScript Dragon (Part 1)

Wednesday, August 3, 2016

This guest post is a part of a short series about Tatyana Goldberg and Guy Yachdav, instructors at Technical University of Munich, and the journey that was inspired by their participation as Google Summer of Code mentors for the BioJS project.

Hello there! We are from the BioJavaScript (BioJS) project which first joined Google Summer of Code (GSoC) in 2014. Our experience in the program set us on a grand open source adventure that we’ll be sharing with you in a series of blog posts. We hope you enjoy our story and, more importantly, hope it inspires you to pursue your own open source adventure.

Tatyana Goldberg and Guy Yachdav, GSoC mentors and open source enthusiasts. Photo taken at the MorpheusCup competition Luxembourg, May 2016.
We came together around the BioJS community, an open source project for creating beautiful and interactive open source visualizations of biological data on the web. BioJS visualizations are made up of components which have a modular design. This modular design enables several things: they can be used by non-programmers, they can be combined to make more complex visualizations, and they can be easily integrated into existing web applications. Despite being a young community, BioJS already has traction in industry and academia.

In early 2014 we decided to apply for GSoC and we were fortunate to have our application accepted on our first try. The experience was extremely positive — the five students we accepted delivered great software and they had a big impact on the BioJS community:
  • The number of mailing list subscribers doubled in less than a month.
  • All five of our accepted students from 2014 became core developers.
  • Students were invited to six international conferences to share their work.
  • Students helped organize the first BioJS conference held July 2015.
  • Most importantly, the students have independently designed BioJS version 2.0 which positioned BioJS as the leading open source visualization library for biological data. 
You can see three examples of the work GSoC students did on BioJS below:


MSAViewer is a visualization and analysis of multiple sequence alignments and was developed by Sebastian Wilzbach. Proteome Viewer is a multilevel visualization of proteomes in the UniProt database and was developed by Jose Villaveces. Genetic Variation Viewer is visualization of the number and type of mutations at each position in a biological sequence and was developed by Saket Choudhary.

We learned a lot in the first year we participated in Google Summer of Code. Here are some of the takeaways that are especially relevant to mentors and organizations that are considering joining the program:
  1. GSoC is a great source of dedicated and enthusiastic young developers.
  2. Mentors need to carefully manage students, listen to them and let them lead initiatives when it makes sense.
  3. Org admins should leverage success in GSoC beyond the program.
  4. Orgs need to find the most motivated students and make sure their projects are feasible.
  5. People want to share in your success, so participation in GSoC can start a positive feedback loop attracting new contributors and users.
  6. Most importantly: the ideas behind GSoC - the love for open source and coding - are contagious and spread easily to larger audiences, especially to students and other people who work in academia. Just try it! 
Our positive experience spurred us to seek out and conquer new challenges. Stay tuned for our next post where we explain how GSoC inspired us to create a popular new class and how we applied data science to Game of Thrones.

By Tatyana Goldberg and Guy Yachdav, BioJS and TU Munich

Omnitone: Spatial audio on the web

Monday, July 25, 2016


Spatial audio is a key element for an immersive virtual reality (VR) experience. By bringing spatial audio to the web, the browser can be transformed into a complete VR media player with incredible reach and engagement. That’s why the Chrome WebAudio team has created and is releasing the Omnitone project, an open source spatial audio renderer with the cross-browser support.

Our challenge was to introduce the audio spatialization technique called ambisonics so the user can hear the full-sphere surround sound on the browser. In order to achieve this, we implemented the ambisonic decoding with binaural rendering using web technology. There are several paths for introducing a new feature into the web platform, but we chose to use only the Web Audio API. In doing so, we can reach a larger audience with this cross-browser technology, and we can also avoid the lengthy standardization process for introducing a new Web Audio component. This is possible because the Web Audio API provides all the necessary building blocks for this audio spatialization technique.



Omnitone Audio Processing Diagram

The AmbiX format recording, which is the target of the Omnitone decoder, contains 4 channels of audio that are encoded using ambisonics, which can then be decoded into an arbitrary speaker setup. Instead of the actual speaker array, Omnitone uses 8 virtual speakers based on an the head-related transfer function (HRTF) convolution to render the final audio stream binaurally. This binaurally-rendered audio can convey a sense of space when it is heard through headphones.

The beauty of this mechanism lies in the sound-field rotation applied to the incoming spatial audio stream. The orientation sensor of a VR headset or a smartphone can be linked to Omnitone’s decoder to seamlessly rotate the entire sound field. The rest of the spatialization process will be handled automatically by Omnitone. A live demo can be found at the project landing page.

Throughout the project, we worked closely with the Google VR team for their VR audio expertise. Not only was their knowledge on the spatial audio a tremendous help for the project, but the collaboration also ensured identical audio spatialization across all of Google’s VR applications - both on the web and Android (e.g. Google VR SDK, YouTube Android app). The Spatial Media Specification and HRTF sets are great examples of the Google VR team’s efforts, and Omnitone is built on top of this specification and HRTF sets.

With emerging web-based VR projects like WebVR, Omnitone’s audio spatialization can play a critical role in a more immersive VR experience on the web. Web-based VR applications will also benefit from high-quality streaming spatial audio, as the Chrome Media team has recently added FOA compression to the open source audio codec Opus. More exciting things like VR view integration, higher-order ambisonics and mobile web support will also be coming soon to Omnitone.

We look forward to seeing what people do with Omnitone now that it's open source. Feel free to reach out to us or leave a comment with your thoughts and feedback on the issue tracker on GitHub.

By Hongchan Choi and Raymond Toy, Chrome Team

Due to the incomplete implementation of multichannel audio decoding on various browsers, Omnitone does not support mobile web at the time of writing.

qLabel: Multilingual content without translation

Tuesday, April 1, 2014

Today we are happy to release qLabel, an open source JavaScript library that looks up and displays the labels of entities marked-up in a Web site in the language of the user. You can use qLabel in any Web document - below are some examples of where it might come in handy.

Some web sites provide content in a very structured form - think of restaurant menus, schedules, images with textual annotations, catalogs, etc. For example, this is a map of the inhabited continents:
Providing this content in different languages is as easy as looking up how all the mentioned entities in the SVG map are named in the other language. If we want to display the content in German, we need to know that South America is Südamerika in German and replace it.
The same works for Chinese:

Or, to take a language that Google Translate does not support yet, such as Uzbek:

The labels that we have used so far are from Wikidata, a sister project of Wikipedia launched in 2012. Wikidata supports more than 300 languages, but there aren't labels for all entities in all languages yet. Let’s take a look at Hindi:
We see that the Hindi name for Australia is still missing. But adding that is as easy as going to the Hindi view of Wikidata for Australia and add the label, and likely by now someone has already fixed it (and that would be visible here if those images above would indeed be embedded SVGs instead of PNG files - see the live map demo). You can improve the content in Wikidata and make more knowledge accessible to everyone.

In these cases, there is no need for intelligent translation algorithms in order to translate the Website: it is enough to look up the label for the mentioned entities in the language of the reader and display them in place. qLabel does exactly that.

The Website author annotates the entities mentioned in the page with unique identifiers, and qLabel looks up the name for these entities in the language requested by the user and displays them. No need to wait until your translation service of choice supports your language, it only depends on the underlying lexicon of entities and the languages they support.

Every entity is marked up with a URI, which is then used to look up the labels in the requested language. Take a look at the examples: the above map, a tournament schedule, a food menu, and tour dates. You can use any URI that supports look-up using Linked Data standards, in particular Google’s Freebase and Wikidata, but you can also publish your own set of entities and labels as RDF or JSON-LD and use them — and at the same time releasing them to the Semantic Web!

Read more about qLabel and how you can use it. Contributions to the code base are more than welcome, the source code is on Github.  Let us know about how you use qLabel!

Thanks and kudos to rdfquery, Wikidata, any23, Freebase, Universal Language Selector, the Wiki Atlas, and the Wikidata Multilingual Picture Dictionary.

By Denny Vrandečić, Ontologist, Google Knowledge Graph 

Wicked Good XPath: a faster JavaScript XPath library

Tuesday, September 4, 2012



We are proud to release Wicked Good XPath, a Google-authored pure JavaScript implementation of the DOM Level 3 XPath specification. We believe it to be the fastest XPath implementation available in JavaScript.

To use Wicked Good XPath, simply download the wgxpath.install.js file and include it on your webpage with a script tag. For example:
<script src="wgxpath.install.js"></script>

Then call wgxpath.install() from your JavaScript code, which will ensure document.evaluate, the XPath evaluation function, is defined on the window object. To install the library on a different window, pass that window as an argument to the install function.

Despite the growing popularity of CSS selectors, XPath remains a useful mechanism to locate elements in an HTML document. It has particularly heavy usage in the context of frontend web testing tools like Selenium and Web Puppeteer. Sometimes there is simply no way to reference an element on the page other than with an XPath expression.

For those who have never used XPath, here is a taste:

On a Google search results page, the XPath expression //li[@class=”g”][3] identifies the third search result. Here is a snapshot from the Chrome extension XPath Viewer showing the third result selected when that expression is used.



A key challenge when using XPath, however, is that it is not natively supported on every browser. Most notably, Internet Explorer does not provide native XPath support for HTML documents. As a result, users turned towards pure JavaScript implementations of XPath. In 2005, Google engineers released AJAXSLT, which included a correct, but slow, XPath evaluator. Running web tests on IE using AJAXSLT was time-consuming.

Fast-forward to 2007, Cybozu Labs released JavaScript-XPath, a new JavaScript XPath implementation that was 10 times faster than AJAXSLT. The web testing tools began using it instead, and life was good... for a while. The JavaScript-XPath quickly fell out of maintenance, and bugs became tough to get fixed. Also, since it wasn't written in Google Closure, it was tricky for us Googlers to integrate into our JavaScript applications. A rewrite was necessary.

However, we went beyond merely porting the library to Google Closure and fixing a couple bugs. We identified some significant additional performance improvements, such that our version runs about 30% faster than the original. On top of that, the Closure compiler was able to minify our code down to a mere 25K, 40% smaller than JavaScript-XPath's 42K. Finally, the code is structured and documented in a way that we believe will make future maintenance quicker and easier.

We would like to thank our two Google interns, Michael Zhou and Evan Thomas, for doing the bulk of the development on this project.

By Greg Dennis and Joon Lee, Google Engineers


Leak Finder: a new tool for JavaScript

Wednesday, August 8, 2012

Leak finder for JavaScript helps web application developers find memory leaks in their JavaScript programs.

In garbage-collected languages, such as JavaScript, you cannot have traditional memory leaks by forgetting to free memory: when all references to an object are dropped, the object is garbage-collected and the memory is freed.

However, JavaScript programs can leak memory by unintentionally retaining references to objects. For example the references can be pointers to objects stored in a data structure in a JavaScript library (e.g., Closure) instead of the application code. If an object is unintentionally retained, all objects it points to are kept alive as well. This will lead to superfluous memory consumption.

Example (using the Closure JavaScript library):

goog.Disposable is an interface for disposable objects. Before dropping the last reference to an object which is an instance of goog.Disposable (or its subclass), the user code is supposed to invoke the method dispose()on the object. This method can release resources, e.g., by disposing event listeners. However, a web application might forget to call dispose() before dropping all the references to an object.

Leak finder can detect such goog.Disposable objects which were not disposed, and print out useful information (such as the stack trace when the object was created) about them. It produces machine-readable output and can be used as a part of test automation.

In order to find leaks, Leak Finder relies on goog.Disposable monitoring mode. The mode gathers all created but not yet disposed instances of goog.Disposable (and its subclasses) into an array goog.Disposable.instances_. This array will keep the objects alive. However, if an object is only kept alive by this array and other Closure data structures, it is a leak, since the user code doesn't contain any pointers to the object, and it cannot call dispose() on it.


Leak finder can be configured to detect other types of memory leaks and it can be used with JavaScript libraries other than Closure.

The Leak finder project page contains instructions for checking out the source code and using the tool.

By Marja Hölttä & Jochen Eisinger, Chrome team (Munich)

How the world was open sourced

Wednesday, December 21, 2011


Once in awhile at Google our illustrators get excited about lasers, Morse code, H. G. Wells’s The War of the Worlds – and then come up with beautiful Google doodles that find their way onto our homepage. Sometimes our programmers also get excited and team up with the illustrators, and that’s how we found ourselves with Google doodles celebrating Les Paul’s guitar, Pac-Man, Jules Verne’s bathyscaphe, and even your own customized turkey that you could then share on Google+.

I’m one of those people who is more comfortable with 80 monospaced characters endlessly repeated than with a paintbrush. Earlier this year I worked with Sophia Foster-Dimino from the Google doodle team on a doodle celebrating Stanisław Lem, my favorite sci-fi writer and philosopher.

 

Just like picking the right paintbrush and palette is important for all our doodles, so is figuring out the right technologies and proper user interface for those we want to make interactive. That’s something I’m personally really excited about and that’s why today I wanted to share that excitement and the entire source code of the Stanisław Lem doodle with you – accompanied with an article explaining HTML5 technologies that we used… or didn’t use:


Please note: We are sharing the code of the doodle under the Apache 2.0 License, but the images and animations accompanying the doodle under the Creative Commons BY-NC-SA 3.0 License. The big difference between those two is that the first one allows commercial re-use, whereas the second one forbids it.

So take it for a spin, play with it, and if you do something interesting, find a flaw, or have a comment – let us know at stanislaw-lem-google-doodle@googlegroups.com. Thanks!

By Marcin Wichary, Senior user experience designer, Chrome

ScriptCover: Javascript coverage analysis tool

Wednesday, October 26, 2011

We are pleased to announce the open source release of a Javascript coverage analysis tool called ScriptCover. It is a Chrome extension that provides line-by-line Javascript code coverage statistics for web pages in real time without user modification of the site. Results are collected when the page loads and continue to be updated as users interact with the page. These results can be viewed in real time through a reporting tool which highlights the executed lines of code for detailed analysis. ScriptCover is useful when performing manual and automated testing and in understanding and debugging complex code.

Short report in Chrome extension popup, detailing both overall scores and per-script coverage.


Sample of annotated source code from the detailed report. First two columns are line number and number of times each instruction has been executed.


We envision many potential features and improvements for ScriptCover, e.g.:
  • support other coverage metrics, e.g. path coverage and condition coverage
  • support richer reports and exporting to HTML and XML
  • submit Javascript coverage statistics to a server and analyze combined statistics for selected users, dates, etc.
  • map user actions to related Javascript code
Want to get involved with ScriptCover and make it better? Join the team! To get started, visit our project page, join the community, read documentation and download the code.

By Ekaterina Kamenskaya, Software Engineer in Test, Google

Introducing Google JS Test

Thursday, September 29, 2011

Google JS Test is a JavaScript unit testing framework that runs on the V8 JavaScript Engine, the same open source project that is responsible for Google Chrome’s super-fast JS execution speed. Google JS Test is used internally by several Google projects, and we’re pleased to announce that it has been released as an open source project.

Features of Google JS Test include:
  • Extremely fast startup and execution time, without needing to run a browser.
  • Clean, readable output in the case of both passing and failing tests.
  • An optional browser-based test runner that can simply be refreshed whenever JS is changed.
  • A built-in mocking framework that requires minimal boilerplate code (e.g. no $tearDown or $verifyAll calls), with style and semantics based on the Google C++ Mocking Framework.
  • A system of matchers allowing for expressive tests and easy to read failure output, with many built-in matchers and the ability for the user to add their own.

See the Google JS Test project home page for a quick introduction, and the getting started page for a tutorial that will teach you the basics in just a few minutes.

By Aaron Jacobs, Google Engineer

Test your app from right to left

Wednesday, November 3, 2010

Can you spot the error in the following webpage?


Unless you are one of the 56 million Internet users who read Arabic, the answer is probably no. But BidiChecker, a tool for checking webpages for errors in handling of bidirectional text, can find it:


Oops! The Arabic movie title causes the line to be laid out in the wrong order, with half of the phrase "57 reviews" on one side of it and half on the other.

As this example demonstrates, text transposition errors can occur even if your web application is entirely in a left-to-right language. If the application accepts user input or displays multilingual content, this data may be in one of the right-to-left languages, such as Arabic, Hebrew, Farsi or Urdu. Displaying right-to-left text in a left-to-right environment, or vice versa, is likely to cause text garbling if not done correctly. So most user interfaces, whether left-to-right or right-to-left, need to be able to deal with bidirectional (BiDi) text.

Handling BiDi text can be tricky and requires special processing at every appearance of potentially BiDi data in the UI. As a result, BiDi text support often regresses when a developer adds a new feature–and fails to include BiDi support in the updated code.

Called from your automated test suite, BidiChecker can catch regressions before they go live. It features a pure JavaScript API which can easily be integrated into a test suite based on common JavaScript test frameworks such as JSUnit. Here's a sample test for the above scenario:


// Check for BiDi errors with Arabic data in an English UI.
function testArabicDataEnglishUi() {



 // User reviews data to display; includes Arabic data.



 var reviewsData = [



 

 {'title': 'The Princess Bride', 'reviews': '23'},


 

 {'title': '20,000 Leagues Under the Sea', 'reviews': '17'},


 

 {'title': 'ستار تريك', 'reviews': '57'} // “Star Trek”



 ];





 // Render the reviews in an English UI.


 var app = new ReviewsApp(reviewsData, testDiv);


 app.setLanguage('English');



 app.render();







 // Run BidiChecker.



 var errors = bidichecker.checkPage(/* shouldBeRtl= */ false, testDiv);



 // This assertion will fail due to BiDi errors!



 assertArrayEquals([], errors);

}

We’ve just released BidiChecker as an open source project on Google Code, so web developers everywhere can take advantage of it. We hope it makes the web a friendlier place for users of right-to-left languages and the developers who support them.

By Jason Elbaum, Internationalization Team

Selenium Ice: A "Cool" New Browser Extension for Internet Explorer

Tuesday, July 1, 2008



I'm happy to announce "Selenium Ice", a new tool for developers who test web applications in Internet Explorer with Selenium. This first version of Ice is limited in scope — it simply adds a new namespace ("window.ice") to every web page, for which developers can write code in C# that is callable from JavaScript. In the future, Ice will be used by the Selenium project team as a building block for providing more native control over the IE browser process. With this native access, Selenium will be able to more reliably catch and close unexpected dialog windows, provide better information when JavaScript errors occur, or do things that require interaction with the operating system, like handling file uploads and downloads. Ice can also serve as a base for linking in faster XPath support in IE — an oft-requested feature among Selenium users.

The Ice project was developed using Visual C# 2008 Express Edition, so no development licensing fees are required beyond access to a machine running Windows. There's an NAnt script for continuous integration, a WiX script for automated creation of a Microsoft Installer (.msi), and a system test script written in Python that can launch IE and ensure that the extension works correctly.

The code is all Open Source and available under the Apache 2.0 license. You can check out our project site on Google Code and may want to join our Selenium Ice discussion group. We always love to hear from you, so let us know what you think of our cool new code!

Dojo Storage

Wednesday, March 12, 2008



The Dojo project is a leading open source Ajax framework for developing advanced web applications in JavaScript. Dojo consists of many modules for powerful cross-browser development, such as modules for offline, modules for graphics, and more. One of these modules is known as Dojo Storage.

Dojo Storage makes it possible to store large amounts of data (hundreds or megabytes of K) on the client-side, way beyond the 4K limit of cookies. Developers are given a simple key/value storage abstraction, similar to a hash table. What makes Dojo Storage unique is that it automatically determines the best way to achieve this. If Google Gears, a small open-source plug-in that teaches current browsers new tricks, is present then this will be used for storage; if the browser supports HTML 5 DOM Storage, such as Firefox 2, then this is used; and finally, if none of the others are available, then a hidden Flash applet is used to store the data permanently. There are even Adobe AIR storage providers (thanks to SitePen and Adobe) if you are running in an AIR environment!

Dojo Storage has been around for a few years. However, when Dojo made the big move to the Dojo 1.0 architecture, the Flash and HTML 5 storage providers broke; plus, new versions of Flash and new browsers made the old design inefficient. I have seriously re-factored the Flash storage system to be much faster and simpler and fixed bugs in the HTML 5 and Gears storage systems. There is now a new storage.js profile build that you can grab and include in your page to easily use Dojo Storage with the three main browser storage providers: Gears, HTML 5, and Flash. The new Dojo Storage will come out as part of the Dojo 1.1 release coming soon.

I've created a screen cast demoing the different storage providers in action:



Enjoy!

Project Hosting Just Keeps on Growing

Wednesday, February 6, 2008



It's been about a year and a half since we launched project hosting on Google Code, and we're now home to over 80,000 projects. From small two-person projects to projects with dozens of contributors, we've got all kinds: utilities, javascript libraries, gadgets, University CS classes, hundreds of Google's own open source projects, and many more.

We continue to work to add more features as well as improve the scalability and reliability of the Google Code hosting service, and in order to keep you more up to date with what we're up to, we've setup shop here in the new Google Open Source blog. You can expect to see posts here that cover new features and bugfixes, reviews of Open Source engineering best practices, and other bits and bobs that we think are relevant to open source software development.

So stay tuned—2008 is looking to be a great year for us.
.