Have you ever wondered what is the secret sauce of Scala.js? What defines Scala.js, above all else, is the overarching will to make it cross-platform. A cross-platform language is both portable—most source code cross-compiles and behaves the same way on multiple platforms—and interoperable—code written in that language can interoperate with other languages on the target platforms. Most multi-platform languages are designed with one of those two properties in mind, and only address the other one as an afterthought. This results in poor integration, difficulty to write portable code, or even the impossibility to use some platform-specific libraries.

This talk shows how Scala.js was designed as a cross-platform language. We show how portability and interoperability drive every design decision, from which language feature to include to performance considerations. We state a criteria of completeness for interoperability, and show how Scala.js satisfies it (with the exception of one JavaScript language feature, still work-in-progress). We give insights about what features are critical for portability, and which ones can be relaxed in the name of performance. In particular, we stress the importance of the ecosystem of libraries: existing assumptions in the ecosystem can force the language designer into reproducing behaviors of Scala/JVM, even when they are accidental.

As a practical exercise, we study the main language changes that made it into Scala.js 1.x wrt. 0.6.x—as well as possible future developments—and how they fit into the cross-platform language design methodology.

Sébastien Doeraene

EPFL, Switzerland

Switzerland

Sébastien Doeraene is a compiler/runtime systems hacker and a Scala enthusiast, now executive director of the Scala Center. He holds a PhD from EPFL, having worked under the supervision of Prof. Martin Odersky, and a master’s degree in computer science engineering from Université catholique de Louvain in Belgium. He is best known as the author of Scala.js. When he is not busy coding, he sings in choirs and a cappella groups such as the Ensemble Vocal Évohé, or rides around on a unicycle.

This paper presents a design for extensible records in Scala satisfying design goals such as structural subtyping, type-safe polymorphic operations, and separate compilation without run-time bytecode generation. Using new features of Scala 3, the design requires only minimal, local changes to the Scala 3 reference compiler Dotty as well as a small library component. Runtime performance is evaluated experimentally using a novel benchmarking suite generator, showing that the design is competitive with reflection-based approaches for structural field access, and excels at immutable extension and update operations.

Olof Karlsson

A3J Consulting, Sweden

Philipp Haller

KTH, Sweden

Germany

Safe object initialization is important to avoid a category of runtime errors in programming languages. In this paper, we provide a case study of the initialization patterns on the Dotty compiler. In particular, we find that calling dynamic-dispatching methods, the usage of closures and instantiating nested classes are important during initialization. Based on the study, we conclude that existing proposals for safe initialization are inadequate for Scala.

Fengyun Liu

EPFL, Switzerland

Switzerland

Aggelos Biboudis

EPFL, Switzerland

Switzerland

Martin Odersky

EPFL, Switzerland

Switzerland

While the Scala type system provides expressive features like objects with type members, the lack of equality checking between path-dependent types prohibits some programming idioms. One such an example is abstract domain combinators in implementing static analyzers. In this paper, we propose to extend the Scala type system with path-equality, and formalize it as a DOT variant, π DOT, which supports records with type members and fields. We show that π DOT has the normalization property and prove its type soundness.

Jaemin Hong

KAIST, South Korea

South Korea

Jihyeok Park

KAIST, South Korea

South Korea

Sukyoung Ryu

KAIST, South Korea

Korea, South

Scala unifies concepts from object and module systems by allowing for objects with type members which are referenced via path-dependent types. The Dependent Object Types (DOT) calculus of Amin et al. models only this core part of Scala, but not many fundamental features of Scala such as strict and mutable fields. Since the most commonly used field types in Scala are strict, the correspondence between DOT and Scala is too weak for us to meaningfully prove static analyses safe for Scala by proving them safe for DOT.

A DOT calculus that can support strict and mutable fields together with constructors that do field initialization would be more suitable for analysis of Scala. Toward this goal, we present κDOT, an extension of DOT that supports constructors and field mutation and can emulate the different types of fields in Scala. We have proven κDOT sound through a mechanized proof in Coq. We present the key features of κDOT and its operational semantics and discuss work-in-progress toward making κDOT fully strict.

Ifaz Kabir

University of Waterloo, Canada

Canada

Ondřej Lhoták

University of Waterloo

Canada

Julia is a young dynamic programming language designed for technical computations. Although it lacks the static typing discipline, Julia is full of types. Multiple dynamic dispatching is the core language mechanism making use of type annotations of method parameters, and residing on surprisingly complicated subtype relation. In previous work we studied one possible algorithmic formulation of the relation — it happens to be full of ugly patches and possibly inefficient procedures. All those irregularities were thoroughly discussed with the language developers and approved as reflecting the actual behaviors encoded in the highly optimized implementation.

We relate Julia’s expressive type language to Scala. We believe that most of the main pieces of the former: invariant type constructors and covariant tuples, existential and union types — can be mapped on Scala or DOT. Yet the latter manages to stay away from disturbing irregularities. We identify key features allowing for that and speculate about the ones which could be added to the system safely.

Artem Pelenitsyn

Northeastern University

Russia

The null reference is both useful and harmful. Null is useful because it allows us to work with values that are only partially initialized. It is harmful because if a binding x has type T, then x can also be null, so operations that should succeed on values of type T might fail on x unexpectedly (i.e. the dreaded NullPointerException).

I will present our work in progress to make null references explicit in Scala. More specifically, I will show a version of Scala where all reference types are non-nullable by default. Nullability can still be expressed via sum types (“x has type T or null”). Because nullability remains implicit in Java, we must have a way for the explicit world of Scala and the implicit world of Java to interact. To this effect, we have designed a type translation from Java types to Scala types that accounts for the fact that Java types remain nullable. For example, in our translation, the Java type “String” is translated into the Scala type “String or null”.

Even if we have non-nullable types, class fields are initialized to null inside constructors. To deal with this case, we plan on integrating our nullability work with a sound initialization scheme in the style of Freedom Before Commitment.

Abel Nieto

University of Waterloo

Canada

Ondřej Lhoták

University of Waterloo

Canada

Parser combinators are a clean and powerful abstraction which can provide reasonably efficient means of parsing a grammar into a form which the programmer desires. They remain close to the domain of grammars whilst at the same time offering enormous flexibility. In Haskell, the Parsec library is a prime example of such a library. However, a direct translation to Scala proves to be unbearably slow. This paper describes the semantics and design of a new library, called Parsley, which retains a close resemblance to Parsec style whilst providing very competitive performance.

Jamie Willis

University of Bristol, UK

United Kingdom

Nicolas Wu

University of Bristol, UK

United Kingdom

Scala heavily relies on a number of object-oriented abstractions to support its feature-rich collections library. There are known techniques that optimize those abstractions away in just-in-time (JIT) compilers, but applying them in the ahead-of-time (AOT) setting is problematic. Profile-guided optimization (PGO) lets AOT compilers apply some of the same optimizations that JIT compilers employ, but it comes at a high complexity cost.

In this paper, we introduce Interflow, an alternative approach towards ahead-of-time optimization of Scala programs which relies on interprocedural flow-sensitive type inference and method duplication. Our evaluation shows that an Interflow-based optimizing compiler built on top of the Scala Native toolchain outperforms existing PGO-based optimizing compilers for Scala.

Moreover, we demonstrate that Interflow and PGO can be combined to achieve further improvements. On our benchmarks, with both Interflow and PGO enabled, the Scala Native toolchain approaches the performance of the HotSpot JVM.

Denys Shabalin

EPFL, Switzerland

Martin Odersky

EPFL, Switzerland

Switzerland

Transparent integration of a domain-specific language for specification of context-free path queries (CFPQs) into a general-purpose programming language as well as static checking of errors in queries may greatly simplify the development of applications using CFPQs. LINQ and ORM can be used for the integration, but they have issues with flexibility: query decomposition and reusing of subqueries are a challenge. Adaptation of parser combinators technique for paths querying may solve these problems. Conventional parser combinators process linear input, and only the Trails library is known to apply this technique for path querying. Trails suffers the common parser combinators issue: it does not support left-recursive grammars and also experiences problems in cycles handling. We demonstrate that it is possible to create general parser combinators for CFPQ which support arbitrary context-free grammars and arbitrary input graphs. We implement a library of such parser combinators and show that it is applicable for realistic tasks.

Ekaterina Verbitskaia

Saint Petersburg State University, Russia

Ilya Kirillov

Saint Petersburg State University, Russia

Ilya Nozkin

Saint Petersburg State University, Russia

Semyon Grigorev

Saint Petersburg State University, Russia

Pattern matching enables inspecting algebraic data types, but typically prevents hiding the implementation of the matched algebraic data type. In Scala, instead, extractors also allow pattern matching on non-algebraic data types and invoking methods on the obtained objects, while partially decoupling API consumers from the API implementation.

But as we show in this paper, pattern matching using extractors is restricted compared to matching against case classes. We argue this violates the appropriate variant of the uniform access principle. To address this problem, we propose a small language extension, which enables defining truly abstract interfaces and freely evolve their implementation.

Nicolas Stucki

EPFL, Switzerland

Switzerland

Paolo G. Giarrusso

EPFL, Switzerland

Martin Odersky

EPFL, Switzerland

Switzerland

Methods in Scala are not first-class values: they cannot be passed around, only called. This is not a fundamental limitation since it’s always possible to wrap a method in a class and create an instance of this class (the wrapped method is usually called apply since Scala will automatically rewrite calls like f(x) to f.apply(x) if necessary). But requiring users to create a new class for every kind of method they wish to wrap in a value would be inconvenient and hard to abstract over, so the language comes built-in with scala.FunctionN classes that each contain an apply method with N parameters, combined with some syntactic sugar and special typing rules this is how Scala represents functions.

However, Scala methods support a rich set of features that are not expressible with functions: they can have type parameters, dependent parameters, implicit parameters, default values, multiple parameter lists, etc. Some of these limitations have now been addressed in Dotty (the experimental compiler slated to become Scala 3), in each case the trick is to find a type to abstract over these functions [1, 2, 3].

In this talk I will describe ongoing work [4] to extend this approach to support polymorphic functions. Unlike implicit functions [2] we cannot simply create a fixed number of classes for them since a polymorphic method can have an arbitrary number of type parameters with arbitrary bounds, and unlike dependent functions [3] we cannot represent them as refinements to scala.FunctionN since a polymorphic method cannot override a monomorphic method. The proposed solution relies on Scala support for structural types, since structural calls on the JVM are not efficient, this is made practical through special type erasure rules. A proof-of-concept implementation is available at [4], the remaining work to be done involves coming up with nice syntactic sugar [5] and designing type inference rules to make polymorphic functions convenient to use.

Guillaume Martres

EPFL, Switzerland

Switzerland

PhD student at EPFL in the LAMP group working on Dotty, the next-generation Scala compiler.

At Twitter, we are using automated code rewrites via Scalafix to enforce code style and to perform library and language version migrations. Such rewrites are practically useful, but they can lead to subtle changes in typechecker behavior around type inference and implicit search. In our experience, such changes may be hard to spot when diffing source files during code review and may not always be covered by tests. Therefore, we believe that there is a need in tools that would detect these changes in an automated fashion.

SemanticDB is a data model for semantic information such as symbols and types about programs in Scala and other languages. This enables an ecosystem of tools that produce and consume semantic information: SemanticDB is currently produced by Scala and Java compiler plugins provided by Scalameta, and consumed by tools like automatic code rewrites, IDEs , and semantic analyzers.

To achieve our goal, we improved SemanticDB to capture synthetic trees that represent inferred type arguments and implicit arguments, and then developed a tool that compares these synthetics before and after automated code rewrites. We used this toolchain to verify the behavior of type inference and implicit search on the core of Twitter Util - a foundational library behind the Twitter stack. Our work has been open-sourced at https://github.com/twitter/rsc.

Max Ovsiankin

UC Berkeley

United States

Eugene Burmako

Twitter, Inc.

United States

A common architectural pattern for complex online systems is a collection of ``microservices'' communicating via RPC interfaces. This arrangement provides autonomy for teams owning services to develop, deploy, and operate their services without centralized coordination. But this autonomy comes at a cost in leverage—the fixed costs of running a service are high, and the desire to avoid them can lead to poor design choices.

In this paper we describe Strato, a system built at Twitter that runs many microservices hosted in a single service platform—trading a little autonomy for a lot of leverage. Hosted microservices are written in a Scala-like DSL that supports transparent concurrency and direct access to Thrift data (the common currency of services at Twitter). The DSL compiles to a concurrency library written in Scala with an arrows-based interface. We detail the design and implementation of the DSL and concurrency library, and evaluate the effectiveness of Strato in improving leverage at Twitter.

Jacob Donham

Twitter, Inc.

Nowadays, there is a clear trend towards standards for communication between developer tools. Language Server Protocol (LSP), a protocol that connects programming language implementations and text editors, has gained strong industrial support and at the time of writing has implementations for many programming languages and editors. Build Server Protocol (BSP) follows in LSP’s tracks with an ambition to define a protocol for communication between language servers and build tools.

While lots of work in the open-source community has been invested in unifying user experience (by codifying commonly used operations like go to definition or find all references), relatively less work went into unifying implementor experience. For example, at the moment, there exist five different LSP implementations for Scala. They all implement the same protocol that works with code, but they all use different data structures to represent that code.

In this talk, we will present SemanticDB - a data model for semantic information such as symbols and types about programs in Scala and other languages. We will discuss our experience of building an ecosystem of developers tools around SemanticDB and using these tools on Twitter source code - arguably the largest Scala codebase on the planet.

Eugene Burmako

Twitter, Inc.

United States

Language tools lead at Twitter, member of the Scala language committee, founder of Rsc, Scalameta and Scala Macros