Vojtech Ruzicka's Programming Blog

Java 13 Text Blocks

Sun, 23 Jun 2019 22:12:03 GMT

Java 13 finally brings support for multi-line string literals after dropping similar functionality from Java 12.

String literals in Java

Declaring a string literal in Java is easy:

String myString = "Behold, a string literal!";

Nothing new, right? Traditional string literals have, however, some inconvenient limitations.

First of all, the whole literal needs to be on just one line. This is ok for simple short strings, but for longer ones, you need to resort to concatenating multiple string literals. If you want also to include line breaks, you need to add \n.

String myString = "This is my string\n" +
        "which I want to be \n" +
        "on multiple lines.";

This is cumbersome and hard to read. The other problem is that various characters cannot be represented directly, but need to be escaped, such as " or \. This makes your strings even harder to read and work with.

String path = "Open \"C:\\Program Files\\Java\\jdk1.8.0_152\"";

This gets especially messy in cases such as regular expressions. How to handle this issue though?

Other JVM languages

As usually, Java is late to introduce a functionality which is already common in many other languages, even those on JVM. Kotlin, Scala or Groovy already support multiline strings literals using """ notation instead of usual ". Look at the following example in Kotlin.

val text = """
This is my cool string,
which I want to span
multiple lines.
It can have backslash \
and even double-quotes "
"""

As you can notice, not only is it multiline, but also you don't need to escape your characters anymore. This is called Raw String literal, which means all the characters are interpreted as is and no escaping is necessary.

Raw String literals in Java

In Java 12, there was originally proposal to include Raw String Literals (JEP-326). The behavior was very similar to the example in Kotlin above. But instead of using """" as in other JVM languages, it used backtick ` notation. The concept was slightly unusual in a way that you could use any number of backticks and the same amount of backticks at the end. This way it was not necessary to escape backticks inside a spring literal no matter how many consecutive backticks were present.

`This uses a single backtick`
``This can contain a backtick `, see?``
```I can use any number of backticks```

One problem with raw string literals is usually indentation. Every character in the multiline raw string literal is interpreted as is - including indentation, which is supposed to be there just for better code readability rather than part of the literal itself. Consider the following example:

  public static void main(String[] args) {
      String myString =
      `This is my string
      which I want to be
      on multiple lines.`;
      System.out.println(myString);
  }

If you print it, all the lines except the first will have indentation present.

This is my string
        which I want to be
        on multiple lines.

You would need to fix this manually, possibly with newly proposed String methods align() and indent(). Not very practical, but it's something you would have to endure if using raw strings.

You can read more about original Raw String Literals proposal in the following article:

Java Raw String Literals

26 November, 2018

#Java

Java finally brings support for raw strings. They can span multiple lines and you don't need to escape special characters. Especially useful for regular expressions.

This proposal was, however dropped before the final release of Java 12 and is now considered obsolete.

While we can expect that for any language feature, there will be a nontrivial volume of "I would have preferred it differently" feedback, in reviewing the feedback we have received, I am no longer convinced that we've yet got to the right set of tradeoffs between complexity and expressiveness, or that we've explored enough of the design space to be confident that the current design is the best we can do. By withdrawing, we can continue to refine the design, explore more options, and aim for a preview that actually meets the requirements of the Preview Feature process (JEP 12).

Multiline String literals in Java

After the withdrawal of the original proposal, the discussion was restarted and resulted in a brand new proposal, which will be introduced in Java 13 as a Preview Feature. Its called JEP 355: Text Blocks (Preview).

It is not supposed to be an implementation of Raw String Literals, but rather multiline String literals, which will still allow escapes, although you'll not be required to use all of them. On top of that, it will not be necessary to handle all the indentation issues, which was necessary with Raw String Literals.

Preview Feature

Text blocks will be only available as a preview feature in Java 13. What does it mean?

A preview language or VM feature is a new feature of the Java SE Platform that is fully specified, fully implemented, and yet impermanent. It is available in a JDK feature release to provoke developer feedback based on real-world use; this may lead to it becoming permanent in a future Java SE Platform.

Before the next JDK feature release, the feature's "real world" strengths and weaknesses will be evaluated to decide if the feature has a long-term role in the Java SE Platform and, if so, whether it needs refinement. Consequently, the feature may be granted final and permanent status (with or without refinements), or undergo a further preview period (with or without refinements), or else be removed.

Such features are shipped in the JDK but are not enabled by default. You need to explicitly enable them to use them. Needless to say, it is not intended for production use, but rather for evaluation and experimentation as it may get removed or heavily changed in a future release.

You'll need to download JDK 13. In IntelliJ, go to File → Project Structure and make sure you have JDK 13 selected under Project SDK. To enable Text Blocks as a preview feature, be sure to select 13 (Preview) - Text blocks under Project language level.

When building your app manually, you need to specify that preview features should be turned on by providing the following params to javac:

javac --release 13 --enable-preview ...

That is for compile-time. At run-time, you just provide --enable-preview

java --enable-preview ...

Text Blocks

Unlike the declined Raw String Literals, Text Blocks are surrounded by three double quotes """", same as in Groovy or Kotlin. It is more in sync with plain String literals and other JVM languages.

String myBlock =  """
                  line 1
                  line 2
                  line 3
                  """

There is no runtime difference between Text Block and String literal. Both result in an instance of String. If they have the same value, they'll get interned as usual and end up as the same instance. Everywhere you can use String literal, you can also use Text Block.

Processing

Unlike regular String literals, Texts block are processed by the compiler in three steps:

Line ends are normalized
Excess whitespace is removed
Escaped characters are interpreted

Line end normalization

Windows and UNIX-based operating systems have different characters to represent line endings.

Windows uses Carriage Return \r and Line Feed \n, while Unix-based systems use just Line Feed. The problem is that Text Blocks use new line characters directly from the source code, instead of using \n such as regular string literals. That means that source code created on Unix would have Strings with different line endings when compiled on windows. The Strings would look identical to the naked eye but would have different line endings.

To prevent this, Java compiler takes all the line endings in Text Blocks and normalizes them to Line Feed \n. What is important is that this is done before evaluating escaped characters. That means that if you explicitly need to include Carriage Return using \r, you can, as it is evaluated after line ending normalization and would not be affected.

Indentation removal

Remember the example with Raw String Literals and indentation? Raw string literals interpret all the characters including indentation. So whitespace which was supposed just to make your source code easier to read actually becomes part of your strings. In the vast majority of the cases, this is not desired behavior.

Fortunately, Java compiler removes unwanted whitespace when compiling Text Blocks.

All the trailing whitespace is removed from the end of the lines.
Leading common whitespace is removed from the start of each line.

What does this mean exactly? Let's look at the following code:

String html = """
    public static void main(String[] args) {
        String html = """
                      <html>
                        <body>
                          <p>Hello, world</p>
                        </body>
                      </html>
                      """;
    }

The HTML code snippet contains a whole lot of whitespace, but it does not really belong to it. It just makes it well aligned within the source file. What is significant is the relative indentation inside the block.

In other words: If every line in the snipped starts with 22 whitespaces, we can ignore them. These 22 spaces are common whitespace prefix, which can be ignored and only what is on top of that should be kept.

Let's replace common prefix with .. All of these spaces will be discarded. Only spaces marked with - will be kept as they exceed the common whitespace prefix.

        String html = """
......................<html>
......................--<body>
......................----<p>Text Blocks are awesome!</p>
......................--</body>
......................</html>
......................""";

The result will be:

<html>
  <body>
    <p>Text Blocks are awesome!</p>
  </body>
</html>

Note that in this step only direct whitespace is removed. If you have whitespace in a form of escaped characters such as \n or \t, it will not be removed.

Escaping

Text Blocks are not raw strings and you can still use escapes. However, you don't need to bother with the most common ones.

New line \n is no longer needed as Text Blocks are multi-line by nature.
You don't need to escape " double quotes as they no longer mark String literal ending

String myBlock =  """
                  First line
                  Second Line with " quotes
                  """

Since the quoting is allowed, you technically can include \n and \", but it is not necessary and it is discouraged. You still do need to escape slash \\. But in general Text Blocks involve a lot less escaping than good old String literals. All the escape sequences, which can be used in String literals, can be also used for Text Blocks. Check the Java spec for the full list.

Interpreting escaped characters as the last of the three steps is important as your escapes are not affected by line ending normalization and whitespace removal.

New String Methods

As part of Text Blocks proposal, there are three new Methods of String class.

translateEscapes() - translates escape sequences in the string escept for unicode ones.
stripIndent() - strips away common whitespace from the beginning of each line.
formatted(Object... args) - Convenience method, equivalent of String.format(string, args)

Conclusion

Text Blocks offer a convenient way of working with multi-line string literals.
To create a Text Block simply surround your string with """".
You can use Text Blocks anywhere you can use String Literals
Line endings are normalized to LF
Extra whitespace is stripped from the beginning and the end of each line. Only relative indentation is preserved.
It is a preview feature in Java 13, which needs to be explicitly enabled
Raw String literals proposal was withdrawn from Java 12; Text Blocks got introduced later instead

Detecting vulnerable dependencies in IntelliJ IDEA

Fri, 07 Jun 2019 22:12:03 GMT

Dependencies with security vulnerabilities

Making sure your application is as secure as possible is very important. Having security vulnerabilities can lead to serious consequences if they are exploited. You should always make sure your own code is as safe as possible.

However, the core code of your application, which you write yourself, is only the tip of an iceberg. These days, the vast majority of the deployed code is not what you write yourself, but rather various third-party libraries and frameworks. Even the simplest web applications usually have hundreds of thousands of lines of code, which are brought with third-party dependencies.

It is very hard to make big and complex frameworks secure. Not only are they very complicated, but they also depend on numerous dependencies of their own. It is not a question IF your libraries contain security vulnerabilities, but rather WHEN they are discovered. Once discovered, they are usually fixed quickly, and the next version is released.

The problematic part is that applications are not that fast with adopting new versions with security fixes. It can take months or even years to upgrade. First of all, you have to know there is a new version. Then you have to know whether to upgrade or not. Were there important security fixes or just some minor bugfixes? Is the fix important enough to justify upgrading cost, especially if there were changes breaking the backward compatibility?

Detecting vulnerabilities automatically

In the past, detecting vulnerable dependencies was difficult. There were public databases of security vulnerabilities, but you had to check them manually. Needless to say that only a minority of developers bothered to check. It was just too much trouble.

Fortunately, later, automatic tools became available, which could scan your dependencies and cross-check with vulnerability databases. One example can be OWASP Dependency Check. Later, even more powerful services appeared, most notably Snyk.

Snyk – Detecting dependencies with known vulnerabilities

22 November, 2017

#Security

How to detect and fix security vulnerabilities in your dependencies using Snyk?

It can be integrated with your Continuous Integration to break the build if critical vulnerabilities are found. What's more, it can even monitor your app, and once a new version fixing your vulnerabilities is released, it will automatically create a Pull request with the new version. How cool is that?

Detection directly in your IDE

Having a check for security vulnerabilities integrated as a part of your CI process is important and necessary, but it comes a bit late in the development pipeline.

It would be much better if you could check locally when adding a new dependency whether there are no security issues. This way, you could discover potential problems much earlier in the process. Snyk offers a CLI tool, which you can use, but for regular checking it can be too cumbersome, and you have to remember to run it every time.

Fortunately, Snyk recently released also IntelliJ IDEA plugin, which monitors your dependencies and can show you security vulnerabilities directly in your IDE.

Snyk plugin

The plugin is available for free for both the Community and Ultimate editions.

The Snyk Plugin is available directly from the plugin marketplace. You can access it using Settings → Plugins.

Once you install it and restart your IDE, a new tab is available:

It shows all the dependencies with known security vulnerabilities, which your app uses. You can expand individual dependencies to explore the transitive dependencies, which introduce the vulnerability.

If there is a new version, which fixes the dependency, Snyk suggests a version to upgrade to, which is very useful. You can click the name of a dependency, and Snyk automatically navigates you to the place, where it is declared in you pom.xml.

Finally, you can click the details link of each vulnerability, which navigates you to a web page in the Snyk vulnerability database with a detailed description of this particular vulnerability. You can then evaluate yourself how critical it is, what can you do to mitigate it a whether it applies in your case or not.

Maven only

Unfortunately, currently (as of 6/2019), the plugin supports only Maven and not Gradle. If you are using Gradle, you can stick With CLI or CI check, which does support it. However, according to Snyk, Gradle support is planned. Hopefully, it will be available soon.

Conclusion

Snyk IDEA plugin offers a simple, yet powerful way to detect security vulnerabilities in your dependencies directly in your IDE. This way, you can discover vulnerable libraries early in the development process without the need to wait for Continuous Integration check. This, however, does not mean you should depend only on the plugin. You should still have CI check in place as a safeguard and continuous monitoring, which Snyk offers, to make sure you discover new vulnerabilities and their fixes once they are available.

IDEA EduTools: Learning directly in your IDE

Sun, 05 May 2019 22:12:03 GMT

With IntelliJ IDEA Edu Tools plugin you can study various programming courses directly in your IDE and work on all the assignments. Alternatively, you can create courses yourself to share with others.

Getting started

Installation

If you already have IntelliJ IDEA installed, all you need to do is to install EduTools plugin. Just go to File → Settings → Plugins → Marketplace and search for EduTools. Then click Install and restart your IDE when finished. The plugin is available both for Ultimate and Community edition.

Alternatively, you can download IntelliJ IDEA EDU, which is basically the Community Edition bundled with the EduTools plugin. If you are using Python, JetBrains also offers Pycharm EDU.

Starting a course

Now, after the plugin is installed, it is time to start using it. All the interactions with the plugin are handled through

File → Learn and Teach

There are several options there, including integration with third-party educational services, such as Coursera, Stepik or Hyperskill. We'll cover these later, let's start simply by browsing available courses in the course repository:

File → Learn and Teach → Browse Courses

Some courses are gray, which means they are not yet approved by JetBrains. To start a course, simply select it and click Join. The join button offers an alternative action to view the course as an Educator. This allows you to access also solutions and tests, which verify whether each task is successfully completed.

Let's open a course in a normal way, as a student for now. Instead of project files as usual, you see all the chapters and the individual tasks contained in them.

As you can see, that IDEA tracks your progress and shows you what is already finished (green), what is failed (red) and what is yet to be completed (empty checkboxes).

You may select and solve individual tasks in any order. After selecting a task, you can see the task code in your editor and the description of what should you change to complete the task.

Once you are finished with your task, you can verify your solution by clicking the Check button. It runs predefined unit tests under the hood and based on the result it lets you know whether you passed or not.

You can either view the solution or just compare the expected output with what you have. Both these features show you a nice version control system-like diff so you can easily identify the differences.

Integrations

The amount of courses available in the repository is still somewhat limited, so it is great that EduTools offers integration with several third-party services, which offer their own courses.

Stepik

Stepik is an online educational platform, where you can either study the available courses or publish your own.

You can log-in in IDEA to your Stepik account. This has several advantages. You'll get access to a broader range of courses and now your progress is automatically synchronized with your Stepik account.

That means that you can study either in IDEA or Stepik web editor and your progress is tracked either way. It also acts as an online backup of your progress, because normally IDEA stores your progress just in a local folder.

You can access Stepik courses in IDEA from:

File → Learn and Teach → Start Stepik Course

Hyperskill

Hyperskill is another service integrated by EduTools. It is actually developed by Stepik. It is a very interesting tool with a slightly different focus. Instead of various independent courses, you build a number of projects. There is a big Knowledge map of various linked topics for each domain, where you can track your progress.

To open a Hyperskill project, go to:

File → Learn and Teach → Start Hyperskill Project

Coursera

Coursera integration is a bit sporadic right now. There's just "Kotlin for Java Developers" by JetBrains, where you can solve your weekly assignments in IDEA. You can access Coursera assignments in: File → Learn and Teach → Start Coursera Assignment.

Checkio

CheckiO offers tasks to improve your programming skills in Javascript and Python. It motivates you through gamification. You solve coding tasks to gain XP and level up to unlock new locations and challenges. There is also a nice social aspect where you can share your solutions and compare with others, have discussions and upvote your favorites.

You can access CheckiO courses through the course browser:

File → Learn and Teach → Browse Courses → Js CheckiO
File → Learn and Teach → Browse Courses → Py CheckiO

Your progress in synchronized with the web as usual.

Creating courses

So far, we've discussed only consuming courses created by someone else. What's great about EduTools is that it allows you to easily create your own tasks and courses. It can be a powerful tool when teaching programming.

This can be useful for hands-on tasks during the lessons, as homework assignments or even self-study material. It's good that it allows students to work with a real IDE with all the support it offers and it will teach them how to use it. They can check their solution automatically without the need to submit their code for review. And as a part of the task description, you can provide any theoretical background information necessary.

New course

The plugin has all you need to create the course directly in IDEA. Simply go to:

File → Learn and Teach → Create New Course

Then fill some basic information about the course:

Currently, you can select one of the supported languages (more are to come):

Java
Kotlin
JavaScript
Python
Scala
Rust

Creating tasks

Now when the course is created it's time to make individual tasks. It's pretty simple. Each task consists of three parts.

First is the description of the task. It is in the form of HTML file, so you can even provide some rich formatting, tables and so on.

The second part is the actual code. There are not two versions of files - starting state and final state. You just provide the final solution, which is expected. Then you can select certain parts of the code and Right Click → Add Answer Placeholder.

That means you mark specific sections of the code as hidden to the students and instead show some text like "TODO implement here".

Finally, you need to provide a way for the students to verify their solutions. This is done via one or more unit tests, which you can provide. As long as they pass, the students can move on to the next task.

Gradle

For Java, Kotlin and Scala each course uses Gradle under the hood and you can access and edit build.gradle file, which is bundled with the course. That means you can utilize the power Gradle gives you. And you are not limited to vanilla Java/Kotlin/Scala, but can also bring various third-party dependencies.

Once you are finished creating the course, it is time to share it with your students. You have basically two options.

You can export your course as an archive, which can be later imported to IDEA by your students.

File → Course Creator → Generate Course Archive

The import is performed from the Course Browser dialog. The other option is to go public and publish your course to Stepik available for everyone:

File → Course Creator → Upload Course to Stepik

Conclusion

EduTools offer you to study various programming courses and solve coding puzzles directly in your favorite IDE. No more clumsy web editors. It integrates well with various third-party educational services and it can synchronize your progress.

You are not limited to third-party courses, though. You can easily create your own courses and use them as a teaching material on your courses.

Analyzing dependencies in IntelliJ IDEA

Tue, 30 Apr 2019 22:12:03 GMT

How to analyze dependencies in IDEA using Dependency Structure Matrix and other tools.

Dependency Matrix

This functionality is available only in IntelliJ IDEA Ultimate, not Community edition.

IDEA offers a useful tool to analyze internal dependencies in your project called the Dependency Structure Matrix. It can be used to analyze not only dependencies between packages, but also individual classes.

This can be very important as tightly coupled classes, and modules or even cyclic dependencies mean that your code is very hard to refactor and reuse.

Dependency Structure Matrix (or DSM for short) can help you visualize your dependencies and look for potential problems. First, make sure to build your project. Then, to launch the analysis go to Analyze → Analyze Dependency Matrix.... You need IDEA Ultimate edition and DSM Analysis bundled plugin enabled.

After the analysis is done, you'll see something similar to this:

To better understand the matrix, look at the following image.

IDEA skips the column labels to save the precious space, but it behaves as if they were there.

Each row and column represent the package. Each cell represents an intersection of a row and column - that is two packages. The number in the cell represents a number of dependencies one package has on another. That means one class from the source package using another class from the destination package. The darker the cell is, the more dependencies. If there are more than 99 dependencies, it shows ... instead of the number.

As long as the dependencies are blue, it is good as they are one way only. Cyclical dependencies would be shown in red, which is not good. But we'll see that later.

Visual aids

The matrix is a powerful tool, which can show a lot of information at once. However, it may be difficult to read at first, if you're not used to working with it. Especially with the columns hidden. Fortunately, there are visual aids, which can help you to better understand the dependencies.

If you hover your mouse cursor over a cell, it shows you with tooltip what is the dependency direction there.

This can be useful, but there is more. You can click any package on the left (not an individual cell with dependency numbers), and IDEA will show you some more useful info.

The row is highlighted, as well as the corresponding column representing the same package. On top of that, you can see that some other packages are highlighted by either yellow on green.

Yellow highlights packages on which the selected package depends
Green marks packages which depend on the selected package

In addition to selecting a whole package, you can also click individual cells. The color coding is the same, there is just one new concept. The purple cell represents dependencies between the same two packages but in the other direction.

We need to go deeper

So far we were working only on package level. This is no doubt useful, but often you need to go into more detail. Which means investigating also dependencies between single classes.

In the DSM view, it is simple, you can expand your packages to see their contents. This way you can see dependencies for all the potential sub-packages or even individual classes.

Cyclic dependencies

So far, we covered only uni-directional dependencies. But something you should be really careful about are cyclic dependencies. In the DSM view, they are marked in red:

You can see that class Owner has 15 dependencies on class Pet, while Pet has 5 dependencies on Owner.

Taking Action

DSM view in IDEA is not only a useful tool to visualize dependencies, but you can also take many useful actions directly from the context menu when right-clicking a cell or package.

You can even refactor your classes directly from the DSM view. If you click a cell with dependencies, you can call Find usages for dependencies. It will look for all the places in one of your classes/packages where the other class/package is used.

Let's look at an example to be more specific. We can find all the places in the service package, where anything from the vet package is used. The results are classified by type:

Now you can inspect the occurrences one by one and check whether you could improve the flow of your dependencies.

Analyzing Maven dependencies

This functionality is available only in IntelliJ IDEA Ultimate, not Community edition.

So far we covered only internal dependencies in your own code. However, you can run into all sorts of trouble also with external dependencies. That is your third-party libraries. It can get messy as you don't have only direct dependencies, but your libraries have dependencies of their own (transitive dependencies). You can encounter many problems such as dependency version conflicts of the same artifact or even cyclic dependencies. Maven offers you to generate dependency tree representation, which can help you in analyzing potential issues. You can simply call:

mvn dependency:tree -Dverbose -DoutputFile=dependencies.txt

It will output your dependency tree in plain text format, which can look something like this (here is the full example):

--- maven-dependency-plugin:3.0.2:tree (default-cli) @ spring-boot-actuator-example ---
com.vojtechruzicka:spring-boot-actuator-example:jar:1.0.0-SNAPSHOT
+- org.springframework.boot:spring-boot-starter-actuator:jar:2.0.3.RELEASE:compile
|  +- org.springframework.boot:spring-boot-starter:jar:2.0.3.RELEASE:compile
|  |  +- org.springframework.boot:spring-boot:jar:2.0.3.RELEASE:compile
|  |  +- org.springframework.boot:spring-boot-autoconfigure:jar:2.0.3.RELEASE:compile
|  |  +- org.springframework.boot:spring-boot-starter-logging:jar:2.0.3.RELEASE:compile
|  |  |  +- ch.qos.logback:logback-classic:jar:1.2.3:compile
|  |  |  |  \- ch.qos.logback:logback-core:jar:1.2.3:compile
|  |  |  +- org.apache.logging.log4j:log4j-to-slf4j:jar:2.10.0:compile
|  |  |  |  \- org.apache.logging.log4j:log4j-api:jar:2.10.0:compile
|  |  |  \- org.slf4j:jul-to-slf4j:jar:1.7.25:compile
|  |  +- javax.annotation:javax.annotation-api:jar:1.3.2:compile
|  |  \- org.yaml:snakeyaml:jar:1.19:runtime

In addition to plain text, it offers various other formats as output, which are better suited for representing dependency graph such as graphml or tgf.

You can specify some other parameters to help you with your investigation, but for larger projects, it can be cumbersome to work with dependency tree output directly.

UPDATE: Be careful with verbose parameter when using maven 3.x as under the hood it still uses Maven 2 algorithm and may give you inconsistent results.

Fortunately, IDEA offers a nice GUI tool to work with Maven dependency graphs. This tool is in IntelliJ for quite some time, but as of version 2019.1, it received some needed enhancements, which make it much more useful in projects with large dependency graphs.

To show the graph, go inside a pom.xml file and press Shift + Ctrl + Alt + U (or ⌥ + ⇧ + ⌘ + U on Mac). Alternatively Right click → Diagrams → Show Dependencies.

You'll see zoomed out dependency graph, where it is not possible to see individual item names unless you zoom in. Looking for individual items manually can be a lot of pain. Fortunately, you can use Find as usual using Ctrl + F.

We are able to locate a specific dependency easily now, but still, a lot of unrelated dependencies making the graph hard to read. Let's look into how to filter the dependencies to get rid of unwanted noise.

The first option is to show only direct neighbors. That is, only direct dependencies of the current item and only items that directly depend on the selected item.

Note that this works not only for single items, but you can select multiple items while holding Shift.

Another way to filter is to display the dependency chain leading from the root to the selected item(s).

IDEA does highlight conflicts for you in red. Still, it may be difficult to find if there is something wrong in a huge dependency tree. Fortunately, you can filter problematic parts only:

In the example above, you can see there is a conflict between JUnit versions. There is an explicit dependency to JUnit 3.8.1 and a different version transitively taken through spring-boot-starter-test.

Maven Helper plugin

If the graphical dependency tree is not your thing, there is an alternative for you. It's Maven Helper Plugin by Vojtech Krasa. It uses hierarchical text representation of dependencies, similar to mvn dependency:tree but with a nice dependency browser.

It is also a good alternative if you are using IDEA Community Edition or have an older version of IDEA, where the Maven dependency graph is way less useful.

To use the Dependency Analyzer offered by the plugin, simply open any pom.xml file. At the bottom of your editor, there is a new tab added by the editor called Dependency Analyzer.

In the left panel, you can browse your dependencies (viewed either as list or tree). The right panel shows how selected dependency got into your application through the dependency chain. You can easily switch the view to show just conflicts.

Conclusion

When you create an application, properly structuring your dependencies is very important as architecture with tight and tangled dependencies can be very hard to maintain, extend and modify. Fortunately, IDEA can help with a Dependency Structure Matrix, which provides a useful graphical view of your internal dependencies.

In addition to that, you need to make sure your external dependencies are covered as well, which you can achieve by utilizing a dependency graph for your dependency management system, such as Maven.

Try with resources in Java

Tue, 23 Apr 2019 22:12:03 GMT

Try with resources offers an easy and foolproof way to make sure all your resources are properly closed. It manages closing automatically instead of explicitly using try-finally.

Traditional try-catch-finally

With traditional try-catch, you can execute a block of code and handle any exceptions which may occur:

try {
    // Do something
} catch (Exception e) {
    // React to exceptions if they occurr
    // in the try block
}

Often it is useful to have also a finally block. It executes after try/catch, no matter whether there was an exception or not. It is very useful, especially when working with resources, which need proper cleanup after being used. When you use some resources, which are locked by your program (such as files or sockets and more), you need to make sure you release them after you're done.

You need to do the cleanup no matter whether all ends well or whether there is an exception.

try {
    // Open a resource
    // Try to use a resource
} catch (Exception e) {
    // React to exceptions if they occur
    // in the try block
} finally {
    // Close the resource when we're done
}

Finally is a nice way to clean everything after we are done, but has its own limitations. There can be multiple places where an exception may occur. That is when opening the resource, using it and even closing. That can result in a lot of boilerplate code, which is hard to read and error prone. We are talking about nested try-catch statements combined with null checks and more.

Most importantly, you need to remember to actually close all the resources and in the right order. That is, resources can have dependencies on each other, which you need to respect.

Fortunately, since Java 7, there is an easier way to manage resources automatically with try-with-resources, which will close all the resources properly for you in the right order.

Try with resources

Using try-with-resources is easy. Instead of just plain:

try {
    // Do something
} finally {
    // Close my resource
}

You create the resource directly after try keyword in parentheses, like this:

try (MyResource resource = new MyResource()) {
    // Do something
} // Resource is automatically closed after this block ends

You basically create a resource, which exists only inside the try block and cannot be accessed afterward. When the try block ends, the resource is automatically closed for you.

Now you ask what kind of resources you can use in try-with-resources and how does JVM know how to close them?

It is actually pretty simple. You can use all the resources, which implement Closeable or AutoCloseable interface. It has only one method called close() which is automatically called for you once the try block finishes. That means it is easy to provide your own custom resources. You can check all the classes implementing Closeable or AutoCloseable in their JavaDoc.

Note that in the example above, we used a simple case of try-with-resources using just try block. Of course, you can include both catch and finally blocks as well. In such case, the catch and finally blocks are executed after all the resource have been closed.

Decompiled example

Let's look at a simple example using try-with-resources, which uses two resources and has just one operation:

try (FileReader fileReader = new FileReader("C:\\foo.txt");
             BufferedReader bufferedReader = new BufferedReader(fileReader)) {

    bufferedReader.readLine();
}

If we decompile this using Fernflower decompiler, we get something similar to this:

FileReader fileReader = new FileReader("C:\\foo.txt");
Throwable var2 = null;

try {
    BufferedReader bufferedReader = new BufferedReader(fileReader);
    Throwable var4 = null;

    try {
        bufferedReader.readLine();
    } catch (Throwable var16) {
        var4 = var16;
        throw var16;
    } finally {
        $closeResource(var4, bufferedReader);
    }
} catch (Throwable var18) {
    var2 = var18;
    throw var18;
} finally {
    $closeResource(var2, fileReader);
}

It is an interesting example of how try-with-resources works under the hood. You can check the official specs for more detail.

Multiple resources

Inside the head of try-with-resources, you can declare more than one resource.

try (FileReader fileReader = new FileReader("C:\\foo.txt");
     BufferedReader bufferedReader = new BufferedReader(fileReader)) {
            // Read some data!
}

All the resources in the () parentheses are automatically closed for you. What's good to know is what is the order of closing. The resources are closed in the reverse order of their declaration to avoid any dependency issues.

That is, if we have three resources like this:

try (resource1; resource2; resource3) {
}

They are created in order of:

resource1
resource2
resource3

However, they are closed in the reverse order:

resource3
resource2
resource1

Java 9 improvements

Try with resources was introduced in Java 7. Until Java 9 you were forced to declare the resources and assign them a value in the parentheses right after try. This is a lot of text and noise, which makes try-with-resources hard to read, especially when using multiple resources.

try (FileReader fileReader = new FileReader("C:\\foo.txt"); 
     BufferedReader bufferedReader = new BufferedReader(fileReader)) {
    // Read some data!
}

Fortunately, since Java 9, you can just reference the name of the existing resource instead of its declaration and assignment. This is much more readable.

FileReader fileReader = new FileReader("C:\\foo.txt");
BufferedReader bufferedReader = new BufferedReader(fileReader);
        
try (fileReader; bufferedReader) {
    // Read some data!
}

It is not shorter, but when you check try-with-resources, you can immediately see which resources it uses without all the noise.

The limitation is, though that all the resources used need to be final of effectively final.

Implementing your own resources

The good news is that you can create your own resources, which can be used in try-with-resources. All you need to do is to implement either java.io.Closeable or java.lang.AutoCloseable.

How are they different? Closeable is older, available since Java 5, AutoCloseable was introduced in Java 7 along with try-with-resources. Since Java 7, Closeable actually extends AutoCloseable.

public interface AutoCloseable {
    void close() throws Exception;
}

public interface Closeable extends AutoCloseable {
    void close() throws IOException;
}

As you can see, they are very similar. The signature of the close() method is different only in the exception thrown. AutoCloseable can throw any Exception while Closeable throws IOException.

Note that when implementing an interface you can change the method signature in a way that it throws a different exception. Or no exception at all. According to the JavaDoc, it is highly recommended to do so in this case. Declare more specific exception in your implementation of close() method or no exception at all if it cannot fail.

There is one more difference, which you cannot see from the method's signature but from JavaDoc only. Closeable is required to be idempotent, AutoCloseable not (although it is highly recommended).

That means you should make sure calling close() multiple times would not cause any trouble.

IntelliJ IDEA integration

As usual, IDEA offers a nice support for try with resources feature. When you are using a resource, which implements AutoCloseable interface, you can surround it with try-with-resources. Just press Alt + Enter to open intention actions popup:

Using the same keyboard shortcut, IDEA allows you to convert traditional try-catch-finally to try-with-resources.

Alternatively, you can do the reverse operation using the same shortcut - convert try-with-resources to good old try-catch-finally.

Conclusion

Try with resources is a useful alternative to traditional try-catch-finally when working with resources, which need to be properly closed. The resource management is automatically handled for you. You can still use catch and finally blocks, as usual, they get executed after the resources are closed.

If you are on Java 9 and later, you don't need to declare your resources directly in the try header, but you can use previously declared resources, which are final or effectively final.

If you want to use your own resources, you can just implement Closeable or AutoCloseable interface and implement the close() method.

Java By Comparison Book Review

Sat, 13 Apr 2019 22:12:03 GMT

This book aims to teach you how to write cleaner, more elegant code in Java. It covers 70 issues of various topics with before and after comparison in a concise, easy to read and understand way.

Authors

The book was written by three authors: Simon Harrer, Jörg Lenhard and Linus Dietz.

The authors are teachers of Java programming courses. When they were evaluating students' assignments, they had to provide the same feedback for the same common issues over and over. A checklist with common issues and proper solutions would help a lot as they could only reference an item from this list when evaluating the assessments. That's how the idea for this book was born. It is a list of common issues and code smells in Java and what can you do to fix them.

Concept

The idea is simple. The book contains 70 independent items. Similar items are grouped into categories, such as comments, testing or exception handling. Each item covers a specific issue, such as "Always Catch Most Specific Exception".

Each item is not just a recommendation or a best practice. Instead, it starts with a problematic snippet of code, which works but has some smells. Then there is a description of what is exactly wrong and why.

On the next page there is a the original snippet, fixed according to a proposed best practice and an explanation of what changed and why is it better.

What is really useful is that the layout of the book consistently keeps each item just on two pages - on the left you can see the problem and on the right the solution. This makes it really easy to compare both at the same time.

I find this format very suitable. First of all, you have specific examples with code, both before and after applying the recommended best practices, which is way easier to understand than just an abstract recommendation. Having the problem presented first also gives you a nice opportunity to think about what's wrong before you are given the answer. This leads to better understanding and deeper learning than just reading recommended best practice outright.

Each item is independent of each other, which makes it easy to skip what you already know. You don't necessarily need to read this book front to back. This makes the book an ideal reference for mentoring and code reviews as you can easily point others just to a specific item.

Even though the items are technically independent, some code examples are further polished over the course of multiple items, which gives a better sense of continuity and understanding how multiple issues relate to each other.

Since each item needs to fit to exactly two pages, the writing style is concise and to the point, while still being easy to understand.

Audience

This book can be very useful to developers with various skill levels for different reasons.

If you are just starting with programming and Java and you're still struggling with basic concepts and syntax, this book may not be for you yet. In fact, at the beginning of the book there is a FizzBuzz test, which is supposed to determine whether you are ready for this book or not. If you are still in very early stages of learning Java and programming, you may want to wait a bit with this book until you grasp basic concepts. Only after learning some basics, you can understand how to write code which not only works, but is also clean and elegant.

You'll get most of this book as a junior Java developer or a person who is switching to Java from another language.

For intermediate devs, this still is useful. You can use the book as a refresher for what you already know and there should still be much to learn from it. Moreover, since the items are independent, you can just skip some items.

As a senior developer, you may not learn much yourself, but the book is a great resource when mentoring others or when doing code review. You can just point them to a specific item when reviewing their code.

The book works with Java 8, so it does not really matter if you're not familiar with all the newer versions of Java.

The book consists of 70 items divided into 9 chapters.

The first two chapters cover basic Java constructs such as iteration, conditions, enums, constants and more. Generally how to write cleaner code, which is less error-prone and easier to understand.

The third chapter covers comments. It guides you through which comments are useless and do more harm than good. Then what should be commented and how. On top of that, some JavaDoc specific advice.

The fourth chapter is dedicated to naming. It is very important to make sure your code is readable and easy to understand. And of course, as we know, "There are only two hard things in Computer Science: cache invalidation and naming things.".

The fifth chapter is focused on excepting handling.

The sixth chapter covers unit testing with JUnit 5. I was pleasantly surprised to see a topic like this in the book as it is not part of the core Java, but of course, it is very important nonetheless. And it is nice to show to beginners early on that testing is important.

The next chapter describes the object-oriented design. The topic is so broad that this book cannot meaningfully cover it on just a few pages.

Lambda expressions and streams are the topics of the eight chapter. It is a nice introduction to the topic with several useful good practices.

The last chapter breaks the usual format and offers a taste of other development areas, which you will encounter in the real world. There is static code analysis, continuous integration, team collaboration, logging, concurrency and more. If you already have some work experience, it may not be anything new. However, it offers a nice intro to the real world development if you are still preparing for your first job.

Conclusion

I really enjoyed this book and I wish I had a book like this when I was starting with Java. It can be a great resource early in your career. It is easy to read, short, easy to understand and can be a good book to read before more heavy-weight titles such as Effective Java or Clean Code, which can be intimidating and harder to grasp when you are just starting.

In your early days, it can also be useful as an introduction (although very brief) to some related real-world topics such as testing, continuous integration or static code analysis. Overall I do recommend this book especially for junior developers, who will benefit from it the most.

Free Samples

To help you decide whether this book is worth getting and reading, there are several free content examples available for free for you to check:

Useful Links

Java 13 Enhanced Switch

Sun, 07 Apr 2019 22:12:03 GMT

Java 12 introduced a whole lot of useful improvements to the good old switch, which makes it way more useful.

There will be syntax changes in Java 13, introducing the new yield statement. This article was updated to reflect it.

Traditional switch

Good old switch is available in Java from the very beginning, and little has changed since then until now. Java's switch follows closely the design of C and C++ including the weird parts.

Fall-through

Most notably, the infamous fall-through behavior. What does it mean? Let's look at a simple switch first:

switch (errorCode) {
    case 404:
        System.out.println("Not found!");
        break;
    case 418:
        System.out.println("I am a teapot!");
        break;
    case 500:
        System.out.println("Internal server error!");
        break;
    default:
        System.out.println("Unknown code!");
}

Nothing fancy, you supply an HTTP result code, and it prints what does it mean. For each code provided, only the corresponding block is executed. As long as you remember to include break.

When a case is matched and executed, the switch automatically continues to the next case unless there is a break statement.

Let's look at an example. Let's pretend we forgot to include a break in the case 404:

switch (errorCode) {
    case 404:
        System.out.println("Not found!");
        // missing break here !!!    case 418:
        System.out.println("I am a teapot!");
        break;
    case 500:
        System.out.println("Internal server error!");
        break;
    default:
        System.out.println("Unknown code!");
}

When we pass 404 to the switch, the first case is a match and the block inside is executed. Because there is no break, the execution continues to the case 418: block even though the input is clearly not 418. In this block, there is break, so the switch finally ends. The output is, however, the following:

Not found!
I am a teapot!

Not what we intended, right? While fall-through behavior can be useful sometimes, more often than not, it is just a result of accidentally missed break. Switch statements are therefore error-prone. In more complicated scenarios, when there is a missing break, you may be in doubt whether it is an error or it was intentional. It can be useful in low-level code, which is performance-sensitive, but in the high-level code, it can be dangerous.

Single value cases

One of the unfortunate limitations of the switch is that in each case, there can be just a single value. This forces you to rely on fall-through if you want to have the same behavior for multiple values.

switch (errorCode) {
    case 400:
    case 404:
    case 405:
        System.out.println("Something is wrong with the request!");
        break;
    case 500:
    case 501:
    case 505:
        System.out.println("Something is wrong with the server!");
        break;
}

What would be much simpler is specifying multiple values per case:

case 400, 404, 405:
    System.out.println("Something is wrong with the request!");

Unfortunately, the old traditional switch does not support this.

Enhanced Switch

Java 12 brought a whole lot of improvements to the traditional switch as Java Enhancement Proposal 325: Switch Expressions (Preview). It solves most of the issues of the traditional switch and is prerequisite of pattern matching, which is to be provided in the future.

Preview Feature

Enhanced switch functionality is, however, currently only available as a preview feature in Java 12. As of Java 13, it will be available as a regular feature.

What does it mean?

A preview language or VM feature is a new feature of the Java SE Platform that is fully specified, fully implemented, and yet impermanent. It is available in a JDK feature release to provoke developer feedback based on real-world use; this may lead to it becoming permanent in a future Java SE Platform.

Before the next JDK feature release, the feature's "real world" strengths and weaknesses will be evaluated to decide if the feature has a long-term role in the Java SE Platform and, if so, whether it needs refinement. Consequently, the feature may be granted final and permanent status (with or without refinements), or undergo a further preview period (with or without refinements), or else be removed.

First, make sure you actually have JDK 12 installed. Then, in IntelliJ IDEA you can enable preview features under File → Project Structure.

Alternatively, if building manually, you need to provide the following params to javac:

javac --release 12 --enable-preview ...

That is for compile-time. At run-time, you just provide --enable-preview

java --enable-preview ...

Multiple values in a case

We already covered that one value per case is problematic and it limits you. Fortunately, with an enhanced switch, there is no longer such limitation. You can simply provide multiple comma-separated values:

switch (errorCode) {
    case 400, 404, 405:
        System.out.println("Something is wrong with the request!");
        break;
    case 500, 501, 505:
        System.out.println("Something is wrong with the server!");
        break;
}

Nice, concise and much easier to read than multiple cases with fall-through, right? So far, so good. Note that we still use break statements.

Switch expression

The traditional switch is a statement. In addition to that, the new switch also adds the possibility of switch expression.

What is the difference? In a nutshell, the statement is imperative to do some logic. The expression returns some value.

Take for example the if-else statement. If some condition is met, it executes if block. If the condition is not met, it executes else block:

if(condition) {
    // do something
} else {
    // do something else
}

The same if-else logic can be achieved with the ternary operator as an expression, which returns some value if the condition is met and a different value otherwise:

x = condition ? 1 : 2;

With the switch it is similar. The traditional switch is a statement. It directs the control flow based on your input. However, you can use the new switch also as an expression. That is, based on your input, it can directly return some value.

String message = switch (errorCode) {
    case 404:
        yield "Not found!";
    case 500:
        yield "Internal server error!";
    default:
        throw new IllegalArgumentException("Unexpected value: " + errorCode);
};

As you can see, we can now use the result of a switch expression and assign it to a variable.

Notice a few differences to the regular switch statement. First of all, it needs to end with a semicolon. Then, there is a new keyword yield. It is similar to the return statement. But for switch expression instead. It marks value to be returned from a switch branch. Because yield terminates the switch expression, you don't need to have break after it.

The yield statement is targeted to be introduced in Java 13. For Java 12, you should use break instead. See History Section below.

Null Pointer Exception Safety

There is another significant difference. Unlike with regular switch, the switch expression needs to cover all the possible input values.

Let's look at the previous example. The errorCode input is an integer. It has, of course, many possible values. Two first cases cover 404 and 500, the rest of the values is covered with the default branch.

That means that all possible integer values are covered. No matter what the input is, the switch always returns something. Or throws an exception explicitly. Let's try to remove the default branch just as a test.

String message = switch (errorCode) {
    case 404:
        yield "Not found!";
    case 500:
        yield "Internal server error!";
    // default:
    //    throw new IllegalArgumentException("Unexpected value: " + errorCode);
};

This will result in the following error:

Error:(11, 26) java: the switch expression does not cover all possible input values

So with switch expression, you have to cover all the possible inputs. Either by providing case for all the possible values (which can be easy for enums) or by providing default case.

This has some nice implications. The regular switch is error-prone when you forget to include one of the values, for example when using enums. Or when you later add another enum item but forget to update your switch statements. This cannot happen with switch expression as you would get compile error. Also, you cannot get null pointer exception as a result of the switch expression.

Switch with arrows

So far we've seen some nice improvements to the good old switch. One of the major nuisances was not covered though. Even with multi-value case blocks, we still had to make sure we included break properly otherwise we could face some nasty fall-through bugs.

Fortunately, the new switch can prevent this. There is a new type of syntax available using ->.

switch (errorCode) {
      case 404 -> System.out.println("Not found!");
      case 500 -> System.out.println("Internal server error!");
}

This syntax can be used for both switch statement and switch expression. In the example above we are using a simple switch statement. In the case of -> switch you don't need to include break and it does not have fall-through behavior. And you can still use multiple values per one case.

It's more concise, easier to read and fool-proof — no more nasty errors.

Now you have two options: if you want to use fall-through behavior, you use the switch with case:, otherwise, you can use the switch with case ->. Whatever approach you use, you need to stick with it for all the case branches in one switch. This results in an error:

// Invalid: both 'case:' and 'case ->' in the same switch
switch (errorCode) {
    case 404 -> System.out.println("Not found!");
    case 500: System.out.println("Internal server error!");
}

Scope

One of the issues with the traditional switch is its scope. The whole switch statement is a single scope. That means if you declare a variable in one of the case branches, it exists in all the subsequent branches until the end of the switch.

switch (errorCode) {
    case 404:
        System.out.println();
        String message = "Not found!";
        break;
    case 500:
        // Cannot declare 'message', it already exists
        String message = "Internal server error!";
        break;
}

This is, of course, needed for the proper working of fall-through behavior. If you want to threat individual case branches as separate scope, you need to introduce a {} block, which is treated as a separate scope:

switch (errorCode) {
    case 404: {
        // This variable exists just in this {} block
        String message = "Not found!";
        break;
    }
    case 500: {
        // This is ok, {} block has a separate scope
        String message = "Internal server error!";
        break;
    }
}

With switch using -> there is no confusion with scoping. On the right side of -> there can be either a single statement/expression, throw statement or a {} block. As a consequence of this, any local variables you want to declare and then use needs to be enclosed in a {} block, which has its own scope, so no more variable clashing.

IntelliJ IDEA support

The good news is that IDEA already has nice support for the extended switch.

In the example below, you can see a quick-fix for replacing the traditional switch with a switch expression. Note that IDEA also automatically adds a default branch, so all the possible inputs are covered.

IDEA also warns you when your switch is not compliant, such as in a case where you don't cover all the possible input values. Moreover, it offers you a quick fix:

If idea detects you have multiple case branches with the same behavior, it will warn you and offer you to merge these branches together:

History

The enhanced switch was originally introduced with Java 12 as JEP-325 - a preview feature, which needs to be explicitly enabled. This was to gather feedback and gain more time for potential further refinement.

For Java 13, there was another proposal introduced - JEP-354, which changed the syntax of returning value from a switch expression.

The original syntax in Java 12 used break keyword followed by the value to be returned.

String message = switch (errorCode) {
    case 404:
        break "Not found!";
    case 500:
        break "Internal server error!";
    default:
        throw new IllegalArgumentException("Unexpected value: " + errorCode);
};

In Java 13, this syntax changes to yield:

String message = switch (errorCode) {
    case 404:
        yield "Not found!";
    case 500:
        yield "Internal server error!";
    default:
        throw new IllegalArgumentException("Unexpected value: " + errorCode);
};

Having a break with return value was a bit confusing and hard to tell apart from regular labeled break. It was harder to visually distinguish a switch statement from switch expression. Now it is much easier to tell them apart:

Switch statement only uses break and never yield
Switch expression only uses yield and never break

The original proposal in Java 13 was break-with, which would be the first hyphenated keyword in Java so far. But it was later replaced with yield.

Future enhancements

As part of the JEP-325 specification, there is also mentioned another improvement, which is not currently implemented (as of Java 12), but may be introduced in the future.

As a target of opportunity, we may expand switch to support switching on primitive types (and their box types) that have previously been disallowed, such as float, double, and long.

Currently, the switch allows input values only of types char, int, byte, short, their object wrappers (Character, Byte, Short, Integer) and String (since Java 7).

Summary

In Java 12 enhanced switch is a preview feature, which needs to be explicitly enabled
In Java 13 enhanced switch is available as a regular feature
You can now use case for multiple values
In addition to the traditional switch statement, you can use switch expression, which returns a value
To return a value from a switch expression you can use:
- break in Java 12
- yield in Java 13
Switch expression must cover all possible input values of given input type
You can use new -> syntax which does not have fall-through and does not require a break
You cannot combine case: and case -> in one switch
Both switch expressions and switch statements can use both case syntax (-> or :)

Javascript async await tutorial

Sun, 31 Mar 2019 22:12:03 GMT

Promises

This article builds on the understanding of the concept of promises in Javascript. If you are not familiar with them or need a quick recap, check the following article.

Javascript promises tutorial

28 March, 2019

#Javascript

Promises are a useful, modern technique in javascript to handle async behavior and a good alternative to callbacks.

Async

Let's assume we have a simple function returning some value

function foo() {
    return 42;
}

Changing it to async function is pretty simple. You just add async at the beginning like this:

async function foo() {
    return 42;
}

Pretty easy right? What did actually change though?

The async function now returns a promise. Instead of returning just 42 as in the previous example, it returns Promise { 42 }, which is fulfilled.

async function foo() {
    return 42;
}

foo().then(result => {
    console.log(result); //42
})

In case that there is an unhandled error in the async function, it returns rejected promise with that error. You can handle it as usual with catch() clause.

async function foo() {
    throw new Error("Oh dear! It's broken!");
}

foo().then(result => {
    console.log(result); // This is not called because of the error
}).catch(error => {
    console.log(error); // Error: Oh dear! It's broken!
});

Async in arrow functions and class methods

Even though our example was on regular function(), you can, of course, use the same concept for arrow functions:

const foo = async (bar) => {
  ...
}

It works even for class methods:

class Foo {
  async bar() {
    ...
  }
}

Await

Await syntax is simple you just put await before your promise:

await myPromise;

What it does is that it waits for the promise to resolve and if successful it returns the value. The execution of the program does not continue to the next line until the promise is settled.

const myPromise = new Promise((resolve, reject) => {
    setTimeout(() => resolve("Completed"), 1000)
});

async function foo() {
    const result = await myPromise; // Wait here until the promise settles
}

As I already mentioned, you can use await only in a function which is marked as async. Otherwise, you get an error:

SyntaxError: await is only valid in async function

One implication of this is that you cannot use await in the top-level code.

Multiple calls

Of course, the example above is rather simple. Where await really shines is multiple subsequent async calls, where each call depends on the value returned by the previous one.

const first = await firstCall();
const second = await secondCall(first);
const third = await  thidrCall(second);

It is a concise, easy to read way to represent multiple async calls. With promises, it would be not so nice chain of then() clauses, which is not so nice. Of course, still much better than chained callbacks.

Error handling

When working with promises, you can provide then() clause, which handles the case where the promise is fulfilled successfully and catch() clause to handle any errors when the promise is rejected. There is even finally() clause which executes no matter whether the promise was fulfilled or rejected.

foo()
.then(successFunction)
.catch(failureFunction)
.finally(doThisNoMatterWhatFunction);

How can you do the same with await though? It just covers the case of then() where the promise is fulfilled. How do you do error handling and possibly finally() functionality?

The good news is that you can use the good old try-catch-finally:

async function foo() {
    try {
        const result = await myPromise;
    } catch (error) {
        // Error handling
    } finally {
        // After try/catch is done,
        // do this whether there was error or not
    }
}

That means - try block executes first. When there is an error, the execution of try block terminates and it jumps to the catch block. No matter whether there was an error or not, in the end, the finally block executes.

Traditional try-catch-finally has some advantages over chained promises with then()-catch()-finally().

It is arguably easier to read. More importantly, it is a plain old traditional JavaScript construct, which everybody knows. You no longer need to use one error handling concept for promises and a different one for non-promise code. Your error handling is unified and cleaner. In one try block you can have mixed async and non-async code.

Waiting for multiple parallel promises

Sometimes instead of resolving async functions one after another, it is useful to execute them in parallel and await until all of them are finished.

With promises, you can use Promise.all(). As input, you provide an array of promises. It returns a single promise, which resolves once all the passed in promises are fulfilled.

Promise.all([promise1, promise2, promise3]).then(doSomething);

Since Promise.all() returns a promise, you can still use await instead of then:

const results = await Promise.all([promise1, promise2, promise3]);

Compatibility

The async-await functionality is well supported in all the major modern browsers. Of course, you'll be in trouble with Internet Explorer. It does not support async await at all, but you can work around this.

Conclusion

Async-await offers an easy way to write asynchronous code as if it was synchronous. It is easier to read and write.

Await can be used before an asynchronous call returning a promise. The execution of the function will wait until the call is resolved and only then continue to the next line.

Since await replaces then().catch() chains, you cannot use the same error handling like with promises. Instead, you should use the good old try-catch-finally.

Await can be only used in functions marked as async. Marking function as async makes it also return a promise. This is done automatically for you, so you don't need to return a promise explicitly.

Javascript promises tutorial

Thu, 28 Mar 2019 22:12:03 GMT

Synchronous execution

Traditionally, the javascript code is executed synchronously. The code is executed from top to bottom, line by line. The execution flow continues to the next line only when the execution of the previous line is fully finished.

console.log("First");
console.log("Second");
console.log("Third");
// First, second, third

Callbacks

However, the model above is not suitable in all cases. Imagine you have a page loaded and need to fetch some date from the backend. If you did this synchronously, all the execution would be suspended until you receive your data. And this could take quite some time. In the meantime, the page would be unresponsive.

In this case, it would be much more convenient to call the server, but continue the execution and react to the result asynchronously whenever the response from your server arrives.

The common traditional approach would be to provide a function, which should be invoked once the server call is done. Such function is called a "callback function".

In other words, you can say: Ok, call the server, I don't care how long it takes, but when you're done, please let me know by calling back this function I provide you with.

You can do this because in Javascript function are first-class objects. You can store functions in variables or pass them as parameters to other functions.

// server url to be requested
const targetUrl = "http://www.example.com/persons";
// Callback function to be called when done
const callbackFunction = function() {
  console.log("All Done!")  
};

// Call the backend
httpClient.get(targetUrl, callbackFunction);
// Now continue and don't wait for response
// Instead, callbackFunction will be executed for us
// When the call is finished

There are many other cases when callbacks can be useful. You can define a callback function to be invoked once a certain time interval passes:

const intervalInMilliseconds = 1000;
setTimeout(function() {
  console.log("One second passed!");
}, intervalInMilliseconds);

Another common use case would be reacting to certain events such as button click, document loaded and so on. You don't know or care when they happen. You just want to be notified by your callback function when the event occurs.

Callbacks and this

Be careful when using this keyword with callbacks. If you are using a function, which is a property of an object, this will refer to the parent object. But when you provide it as a callback, it will no longer point to its object, which can lead to unexpected behavior. To learn in detail about issues with this, check the following article:

Javascript: Uncovering mysteries of ‘this’ keyword

13 February, 2018

#Javascript

This is a very straightforward concept in other languages. Not so in javascript. It can point to pretty much anything depending on the context.

Promises

Promises offer an alternative approach to callbacks when working with asynchronous calls.

A function can return promise object, which wraps the async call an can be used to define what should be done when the promise is finally successfully resolved. The promise object contains the then method, which can be used to define a function, which should be called once the promise successfully finishes. Let's look at the previous example using promises:

// server URL to be requested
const targetUrl = "http://www.example.com/persons";
// Callback function to be called when successfully finished
const callbackFunction = function() {
  console.log("All Done!")  
};

// Call the backend. httpClient.get returns a promise
httpClient.get(targetUrl).then(callbackFunction);

So far it is quite similar to callbacks. Of course, something can go terribly wrong when calling the server. If your URL is not valid, you can get 404 not found, or there can be 500 internal server error. Fortunately, in addition to what happens when everything goes well, you can define another function which should be called in case the promise is not successful. You can use catch() and chain it after then:

// server URL to be requested
const targetUrl = "http://www.example.com/persons";
// Callback function to be called when successfully finished
const successFunction = function() {
  console.log("All Done!")  
};

// Function to be called when there is a failure
const failureFunction = function() {
  console.log("Oh dear! Something went terribly wrong!")  
};

httpClient.get(targetUrl).then(successFunction).catch(failureFunction);

In case the promise is resolved successfully as expected, the function in then() will be called, otherwise, when there is an error, function in catch() will be called.

Finally

In addition to then() and catch() there is a useful clause called finally(). It is run after the promise is settled, no matter whether it was successful or not. It is handy to prevent duplicated code, which would be otherwise both in catch() and then(). If you have code which should be executed after the promise finishes no matter what, it belongs to finally().

httpClient.get(targetUrl)
.then(successFunction)
.catch(failureFunction)
.finally(doThisNoMatterWhatFunction);

Promise state

Each promise can be in one of the following states:

settled: Promise is finished, either fulfilled or rejected
rejected: The action was not successful, promise failed catch function is was called if present. The promise is finished.
fulfilled: Promise action was successful, then function was called. The promise is finished.
pending: The promise is not finished, still in progress.

Creating promises

You can easily create a promise object by calling new Promise() and supplying a single function as an input parameter. That function should have two parameters - resolve and reject.

new Promise(function(resolve, reject) {
  ...
});

You should call resolve() and supply a return value when the promise is fulfilled. In case something goes wrong, you call reject() instead, and you can supply an error describing what happened.

new Promise(function(resolve, reject) {
  if(everithingOk) {
      resolve("It works!");
  } else {
      reject(new Error("Something went horribly wrong!"))
  }
});

A promise constructed in this way should be returned from your function:

function myFunctionUsingPromises(input) {
    return new Promise(function(resolve, reject) {
             if(input === 42) {
                 resolve("It works!");
             } else {
                 reject(new Error("Something went horribly wrong!"))
             }
           });
}

Automatic rejection

In the example above, the promise is rejected explicitly by calling reject(). However, this is not the only case where promise can be rejected. When there is an error while running the promise code, the exception is automatically caught and reject() is called for you.

new Promise(function(resolve, reject) {
              throw new Error("Oh dear! Something terrible happened!");
           }).catch(function(error) {
               //The error thrown is properly caught
               console.error("Error during promise:");
               console.error(error);
           });

As you can see, the error is properly caught and can be handled inside the catch() clause. So if you don't need special error handling logic directly inside the promise, you don't necessarily need to include try-catch in it.

Multiple promises

So far we worked only with single promises. In some cases, it is just enough. However, often you need to work with more promises which are somehow related to each other. Let's look at specific examples.

Serial

The most common use case is probably a scenario where you need to run multiple promises one after each other. And each of them is dependent on the last one. That is, you cannot run the second promise until the first one is resolved.

Let's say you need to fetch some data from the server and only after you receive the data, you can execute a second server call, which uses the data previously obtained.

Fortunately, this is easy with promises. All you need to do is to chain them one after another using then.

new Promise(function(resolve, reject) {
    resolve(42);
}).then(function(result) {
    return result + 8;
}).then(function(result) {
    return result*2;
}).then(function(result) {
    // Result is (42 + 8)*2 = 100
    console.log("The total result is: "+result);
});

Easy, right? Not only easy to write, but also easy to read. Especially when you use arrow functions like this:

new Promise((resolve, reject) => {
    resolve(42);
}).then((result) => {
    return result + 8;
}).then((result) => {
    return result*2;
}).then((result) => {
    console.log("The total result is: "+result);
}).catch(error => {
    //Error handling
});

As you can see in the example above, you can still use catch to handle errors. What's even better, the catch will handle errors which happen inside of ANY of the then blocks.

Be aware that to achieve the desired "one after another" behavior, you need to chain promises. If you just create multiple independent then clauses it does work differently:

const promise = new Promise((resolve, reject) => {
    resolve(42);
});

promise.then((result) => {
    return result + 8;
});

promise.then((result) => {
    return result*2;
});

promise.then((result) => {
    console.log("The total result is: "+result);
});

Any guess what will happen? Instead of running promises one after another, you just provide multiple handlers to the same promise. Each of them will be executed independently after the promise resolves. The total result will still be 42.

Parallel - all

Another scenario is when you have multiple promises and you need to run them all. Only after ALL are done, you want to do something. However, the promises are not dependent and can run in parallel with each other.

Promise.all([promise1, promise2, promise3]).then(doSomething);

You just provide an array of promises to Promise.all().

More detailed example including promise declaration:

const promise1 = new Promise(resolve => setTimeout(() => {
    console.log("Promise 1 resolved!");
    resolve(1)
}, 1000));
const promise2 = new Promise(resolve => setTimeout(() => {
    console.log("Promise 2 resolved!");
    resolve(2)
}, 500));
const promise3 = new Promise(resolve => setTimeout(() => {
    console.log("Promise 3 resolved!");
    resolve(3)
}, 1500));

Promise.all([
    promise1,
    promise2,
    promise3
]).then((results) => {
    console.log("All DONE!")
    console.log(results);
});

If you run the code above, you'll get the following output:

Promise 2 resolved!
Promise 1 resolved!
Promise 3 resolved!
All DONE!
[ 1, 2, 3 ]

You can see that the result is an array of the results returned by each individual promise.

What's important to note is that even though promises resolve in order 2,1,3 (because of the different timeouts set), the result array is in the same order as what you put in Promise.all(), that is 1,2,3. Otherwise, it would be hard to tell which result belongs to which promise.

Parallel - any

The last scenario is when you want to call multiple similar promises, but you are interested just in whatever comes first.

The syntax is similar to Promise.any(), but you use Promsie.race() instead.

Only the result of the first resolved promise is passed into then() but all the promises still resolve. But their result is just ignored.

let promise1 = new Promise(resolve => setTimeout(() => {
    console.log("Promise 1 resolved!");
    resolve(1)
}, 1000));
let promise2 = new Promise(resolve => setTimeout(() => {
    console.log("Promise 2 resolved!");
    resolve(2)
}, 500));
let promise3 = new Promise(resolve => setTimeout(() => {
    console.log("Promise 3 resolved!");
    resolve(3)
}, 1500));

Promise.race([
    promise1,
    promise2,
    promise3
]).then((result) => {
    console.log("All DONE!")
    console.log(result);
});

The result is then:

Promise 2 resolved!
All DONE!
2
Promise 1 resolved!
Promise 3 resolved!

Beware that if the first promise results in an error and is rejected, then() is not executed.

Compatibility

Promises are well supported in all the modern browsers as you can see in the table below:

Well, except for Internet Explorer, of course. For the up to date compatibility list see this table on Can I Use.

For support in IE, you need to use a polyfill.

Async await

While promises are easier to write, read and maintain than callbacks, they still have their issues, especially in more complex cases of promise chains. Async-await is an interesting alternative to plain promises, which allows you to have asynchronous code, which reads as synchronous code, is simple and concise. Check the following post for detailed explanation.

Javascript async await tutorial

31 March, 2019

#Javascript

Async await allows you to work with asynchronous code in cleaner and more convenient way.

Destructuring objects and arrays in JavaScript

Sun, 03 Feb 2019 22:12:03 GMT

Destructuring assignment

The destructuring assignment allows you to assign items of arrays or properties of objects to separate variables. Let's look into more detail how this works.

Array destructuring

Let's assume we have an array and we want to assign its contents into separate variables. Without destructuring, you would need to do something like this:

let myArray = [1, 2, 3]; 
let a = myArray[0];
let b = myArray[1];
let c = myArray[2];

With array destructuring assignment, it is much easier:

let myArray = [1, 2, 3];
let [a, b, c] = myArray; // a=1, b=2, c=3

Much better, right? The first item from array gets assigned to the first variable, the second item in the array to the second variable and so on.

In the example above, we declared new variables and assigned to them, but you can use existing ones.

Of course you don't need to use all the items from the source array:

let myArray = [1, 2, 3];
let [a, b] = myArray; // a=1, b=2

Default values

On the other hand, if there are not enough items in the array, only some of the variables get assigned. The rest remains undefined as if you would declare a variable and not assign it.

let myArray = [1];
let [a, b, c] = myArray; // a=1, b=undefined, c=undefined

You can provide default values to the variables in case there is not enough items, so they have some value as a fallback:

let myArray = [1];
let [a=2, b=4, c=6] = myArray; // a=1, b=4, c=6

Because it is called destructuring, you might think that items are actually removed from the source array. This is, however, not the case, the source array remains unchanged. The same also applies to destructuring objects.

Ignoring items

If you want to skip certain items, you can do it like this:

let myArray = [1, 2, 3];
let [a, , b] = myArray; // a=1, b=3

Swapping variables

The traditional approach to swapping variables involves using a third temporary variable:

temp = a;
a = b;
b = temp;

With destructuring, variable value swap is as easy as this:

[x, y] = [y, x];

Assigning the rest of the items

You can assign just a first few items from the source array and put all the unassigned items in a new array:

let myArray = [1, 2, 3, 4, 5, 6];
let [a, b, ...others] = myArray; // a=1, b=2, others = [3, 4, 5, 6]

The ... syntax with its various uses is described in detail in the following article:

Javascript spread operator and rest parameters (...)

30 January, 2019

#Javascript

What do three dots (...) mean in javascript? They can be used for various different purposes.

Nested arrays destructuring

When you have an array, which contains other arrays, you can still use destructuring to get to the items in the nested array:

let myArray = [1, 2, [3, 4], 5];
let [first, second, [third, fourth], fifth] = myArray;
// first = 1, second = 2, third = 3, fourth = 4, fifth = 5

Destructuring iterables

All the examples above used arrays as a source for destructuring. Actually, this is just a special case as you can use any iterable such as string. In that case each character gets assigned to a single variable:

const person = "John Doe";
let [first, second] = person; // first = "J", second = "o"

Or you can even use regular expression matches:

let pattern = /(\w+)/g; // Match whole words
let string = "The quick brown fox jumps over a lazy dog.";
let results = string.match(pattern);
// results = ["The", "quick", "brown", "fox", "jumps", "over", "a", "lazy", "dog"]
let [first, second] = results; // first = "The", second = "quick"

Object destructuring

Object destructuring works in a similar way to array destructuring with a few distinctions.

// Simple assignment
let john = {name: "John Doe", age: 42, gender: "Male"};
let {name, age} = john;

// Default values
let {name, age, favoriteColor = "Black" } = john; 

// Other unused properties assigned to a new object
let {name, age, ...others} = john;

Changing variable names

Unlike arrays, where the assignment is determined by order, here it is by variable name matching object's property name. That means that is assigning to a variable called name it will use value of a property name from the source object.

However, if you want, you can make the name different if you want:

let john = {name: "John Doe", age: 42};
let {name: newName, age: newAge} = john;
// newName ="John Doe", newAge = 42

That can be particularly useful when working with object property names, which are not valid variable names. For example, this is a valid object:

{"my-property": 42}

But my-property is not a valid variable name. Fortunately, we can assign different variable name, which is valid using the approach above:

// Not valid syntax: my-property is not valid variable name
// But it is valid object property name
let {my-property} = {"my-property": 42};

// But we can change my-property to valid variable name
let {"my-property": myProperty} = {"my-property": 42};
// myProperty = 42

Dynamic property names

When working with plain objects, you can access their properties by property name like person.name or person["name"]. What's more interesting, you can also use a variable in place of property name - person[myVariable]:

let person = {name: "John", age: 42, hobby: "Javascript"};

let propertyKey;
if(something) {
    propertyKey = "age";
} else {
    propertyKey = "name";
}

let foo = person[propertyKey]; // foo = 42 or "John"

You can use similar approach when destructuring:

let person = {name: "John", age: 42, hobby: "Javascript"};

let propertyKey;
if(something) {
    propertyKey = "age";
} else {
    propertyKey = "name";
}

let {[propertyKey]: foo} = person; // foo = 42 or "John"

Usage in iteration

Destructuring can be useful when iterating over multiple objects. You can easily extract just the properties you are interested in:

let persons = [{name: "John", age: 42, hobby: "Javascript"}, {name: "Jane", age: 24}];

for(let {name, age, hobby = "Unknown"} of persons) {
    console.log(name);
    console.log(age);
    console.log(hobby);
}

It is not only concise when accessing the current object's properties, but also it allows you to define default values of missing properties easily.

Assignment to existing variables

When destructuring arrays, it is not different whether you declare your variables and immediately assign to them or whether you assign to previously declared variables:

// Declaration and assignment
let [foo, bar] = [1, 2];

// Declaration and assignment separately
let foo, bar;
[foo, bar] = [1, 2];

With objects it is different:

// Declaration and assignment: perfectly fine
let {foo, bar} = {foo: 1, bar: 2};

//SyntaxError: Unexpected token =
let foo, bar;
{foo, bar} = {foo: 1, bar: 2};

What happened here? Curly braces in {a, b} get interpreted as a declaration of a block rather than object destructuring assignment. Fortunately, you can wrap the assignment in regular braces to make it work:

({foo, bar} = {foo: 1, bar: 2});

Nested object destructuring

Similar to nested array destructuring, you can destructure even items in the nested objects of the source object:

let john = {
        name: "John Doe",
        age: 42,
        skills: {
            cooking:9,
            javascript: 1,
        }
};

let {age, name, skills: {cooking, javascript}} = john;
// name = "John Doe", age = 42, cooking = 9, javascript = 1

Of course, when your object contains arrays as property values, you can destructure them as well. You can even combine both:

let john = {
    name: "John Doe",
    age: 42,
    skills: {
        cooking:9,
        javascript: 1,
    },
    hobbies: ["Soap operas", "Internet trolling"]
};

let {age, name, skills: {cooking, javascript}, hobbies: [hobby1, hobby2]} = john;
// name = "John Doe", age = 42, cooking = 9, javascript -=1
// hobby1 = Soap operas, hobby2 = Internet trolling

Destructuring function parameters

Imagine you have a function, which instead of various individual parameters accepts an object with multiple properties:

const myImage = {
    x: 100,
    y: 200,
    source: "http://example.com/image.png",
    grayscale: false
};

function drawImage(imageConfig) {
    console.log("Drawing image:");
    console.log("Position: " + imageConfig.x + "," + imageConfig.y);
    console.log("Source image location: " + imageConfig.source);
    console.log("Grayscale: " + imageConfig.grayscale);
}

drawImage(myImage);

This is often useful, especially if you have many configuration properties and some of them are optional. However, it has some disadvantages. You need to access all the properties through the source object like this: imageConfig.grayscale. Alternatively, you need to declare variables and assign properties to them. With optional properties, it is even worse as you need to handle manually default values if a property is not present. Fortunately, you can use object destructuring to create variables from the input object:

function drawImage({x, y, source, grayscale}) {
    console.log("Drawing image:");
    console.log("Position: " + x + "," + y);
    console.log("Source image location: " + source);
    console.log("Grayscale: " + grayscale);
}

But of course, it is even more useful with default values:

function drawImage({x = 0, y = 0, source, grayscale = false}) {
    console.log("Drawing image:");
    console.log("Position: " + x + "x" + y);
    console.log("Source image location: " + source);
    console.log("Grayscale: " + grayscale);
}

Conclusion

Destructuring is a useful tool which allows you to break down complex structures such as arrays and objects to simple parts. The syntax is much more concise than the traditional approach, especially when handling situations such as default values.

Javascript spread operator and rest parameters (...)

Wed, 30 Jan 2019 22:12:03 GMT

Variable number of parameters

Usually, when working with functions in JavaScript, there is a fixed number of parameters your function uses and supports. If you want to calculate the square root of a number using Math.sqrt(x) you always need just one parameter for input. Some functions may require more parameters. Others have more parameters, but you can safely omit last few optional ones. In any of these cases, all the parameters are declared as a part of the function signature:

function doSomething(first, second, third) { 
    // Do something cool
}

However, what if the number of parameters is not known beforehand and can be pretty much anything?

For example, let's take Math.min() and Math.max(). It does work for any number of parameters no matter whether it is 0, 2 or 30.

Math.Min(); // Infinity
Math.Min(1); // 1
Math.Min(1,2,3,4,3,1,5,7,9,0,-9,18,37,81); // -9

How would you write such a function?

Arguments - traditional approach

When you need to work with a variable number of parameters in a function, traditionally your only option would be arguments object.

Inside the body of each function, there is arguments object available, which is array-like structure, which contains all the arguments passed into the function. You can access each passed-in argument with a zero-based index and you can check the length of the parameters as you would with any array.

function doSomething() { 
    arguments[0]; // "A"
    arguments[1]; // "B"
    arguments[2]; // "C"
    arguments.length; // 3
}

doSomething("A","B","C");

Arguments limitations

Note that the arguments object is array-like, not a full-fledged array. That means that useful methods, which arrays have are not available. You cannot use methods such as arguments.sort(), arguments.map() or arguments.filter(). The only property you have is length.

Another limitation is that fat-arrow functions don't have arguments object available. In case you would use arguments in arrow function, it would refer to the arguments of the enclosing function (if there is one).

Rest Parameters

Fortunately, since ES6, the arguments object is no longer the only way how to handle variable parameters count. ES6 introduced a concept called rest parameters.

What it means is that you can put ... before the last parameter in the function.

function doSomething(first, second, ...rest) {
    // do something cool
}

Now, in the example above, you can access the first two named parameters as usual. However, all the other arguments passed to the function starting with third are automatically collected to an array called as the last parameter ("rest" here).

function doSomething(first, second, ...rest) {
    console.log(first); // First argument passed to the function
    console.log(second); // Second argument passed to the function
    console.log(rest[0]); // Third argument passed to the function
    console.log(rest[1]); // Fourth argument passed to the function
    // Etc.
}

If you pass less than three parameters, rest will be just empty array.

Unlike arguments object, rest parameters give you a real array so that you can use all the array-specific methods. Moreover, unlike arguments, they do work in arrow functions.

let doSomething = (...rest) => {
    rest[0]; // Can access the first argument
};

let doSomething = () => {
    arguments[0]; // Arrow functions don't have arguments
};

In addition to the advantages above, there is one which makes the rest parameters superior to arguments. Rest parameters are part of the function signature. That means that just from the function "header" you can immediately recognize that it uses rest parameters and therefore accepts variable number of arguments. With arguments object, there is no such hint. That means you have to read the method body or comments to be able to tell that it accepts a variable number of parameters. What's also important is that IDEs can detect rest parameters in the signature and help you better.

Limitations

Rest parameters take all the remaining arguments of a function and package them in an array. That naturally brings two limitations. You can use them max once in a function, multiple rest parameters are not allowed.

// This is not valid, multiple rest parameters
function doSomething(first, ...second, ...third) {
}

And you can use rest parameters only as a last parameter of a function:

// This is not valid, rest parameters not last
function doSomething(first, ...second, third) {
}

Spread operator

What's a little bit confusing is that the same three dots ... used in JavaScript for the rest parameters are also used for an additional purpose. It is called spread operator. Its usage is almost the opposite of rest parameters. Instead of collecting multiple values in one array, it lets you expand one existing array (or other iterable) into multiple values. Let's look at various use cases.

Function calls

Let's assume we have an array with three items and a function, which requires three input parameters.

let myArray = [1, 2, 3];

function doSomething(first, second, third) {
}

How do you pass three values in our array as a three separate arguments to the doSomething() function? A naive approach would be something like this:

doSomething(myArray[0], myArray[1], myArray[2]);

It is obviously not very nice, difficult for a large number of parameters and works only if we know their number before. Let's try something else. Before spread operator, this used to be the way to call a function and pass an array as separate parameters:

doSomething.apply(null, myArray);

The first parameter of the apply function is what's supposed to be the value of "this". The second one is the array we want to pass into the function as arguments.

With spread operator, you can achieve the same using:

doSomething(...myArray);
// Is equivalent to 
doSomething(myArray[0], myArray[1], myArray[2]);

Note that it works with any iterable, not just arrays. For example, using the spread operator with string will disassemble it to the individual characters.

You can combine this with passing individual parameters. Unlike rest parameters, you can use multiple spread operators in the same function call and it does not need to be the last item.

// All of this is possible
doSomething(1, ...myArray);
doSomething(1, ...myArray, 2);
doSomething(...myArray, ...otherArray);
doSomething(2, ...myArray, ...otherArray, 3, 7);

Array literals

The spread operator can also be used when creating an array using an array literal. This way you can insert elements from other arrays (or iterables such as strings) at a specific location.

let firstArray = ["A", "B", "C"];
let secondArray = ["X", ...firstArray, "Y", "Z"];
// second array is [ "X", "A", "B", "C", "Y", "Z" ]

By using spread operator in an array literal, we say: the second array should have X as the first element, then all the elements from the firstArray, no matter what is their number. And then the last two elements will be Y and Z.

Merging arrays

Of course, you can use spread operator in an array literal multiple times. This can be useful for example for merging several existing arrays, which would be more difficult using traditional imperative approach:

let mergedArray = [...firstArray, ...secondArray, ...thirdArray];

Copying arrays

Sometimes it is also useful to create a copy of the existing array with the same items as the original. It is easy with the spread operator.

var original = [1, 2, 3];
var copy = [...original];

Object literals

Very similar to array literals is the usage of the spread operator in object creation with object literals. You can take properties of another object and include them in a new object created by an object literal. This feature is available since ES2018.

let firstObject = {a: 1, b: 2};

let secondObject = {...firstObject, c: 3, d: 4};

console.log(secondObject); // { a: 1, b: 2, c: 3, d: 4 }

Be aware that spread takes only own (not inherited) and enumerable properties of an object, other properties are ignored.

Shallow copy

The use cases are pretty much the same as with arrays. You can also merge and clone objects.

let clone = {...original};

This can be a nice alternative to cloning objects using Object.assign(). Note that it is a shallow copy. A new object is created, but the cloned properties are still the original and not clones.

Prototype lost

When cloning an object using the approach above, be aware that the prototype of the original object is not preserved. It just copies properties of the source object and creates a brand new object using object literal, which has a prototype of Object.prototype.

Property conflicts

What happens though when you introduce a property with the spread operator which already exists in the object? This does not result in an error. If there are multiple object properties with the same name and different values, the latest one wins.

let firstObject = {a: 1};
let secondObject = {a: 2};

let mergedObject = {...firstObject, ...secondObject};
// a: 2 is after a: 1 so it wins
console.log(mergedObject); // { a: 2 }

Updating immutable objects

The behavior where the later declared property with the same name wins can be utilized when updating immutable objects. When you're working with immutable objects or don't want to directly mutate objects, you can use spread operator to create a new object as an updated variant of the original.

let original = {
      someProperty: "oldValue", 
      someOtherProperty: 42
    };

let updated = {...original, someProperty: "newValue"};
// updated is now { someProperty: "newValue", someOtherProperty: 42 }

Because the original object contains someProperty and it is then used once more, the last usage wins and the new value is used. Original object is not changed in any way and a new object is created.

Destructuring assignment

In short, the destructuring assignment is a way to assign properties of objects or values from arrays to distinct variables.

Array destructuring

Let's assume we have an array with three items and we want to assign these items into three separate variables.

let myArray = [1, 2, 3];
let a,b,c; // We want to assign a=1, b=2, c=3

Ths can be done easily by array destructuring assignment like this:

let myArray = [1,2,3];
let [a, b, c] = myArray;

console.log(a); // 1
console.log(b); // 2
console.log(c); // 3

But how is this connected to the three dots syntax? Remember rest parameters? In the same way as all the remaining arguments are packed into an array, we can assign all the remaining items in the array destructuring assignment into a new array variable:

let myArray = [1, 2, 3, 4, 5];
let [a, b, c, ...d] = myArray;

console.log(a); // 1
console.log(b); // 2
console.log(c); // 3
console.log(d); // [4, 5]

Object destructuring

Object destructuring is very similar to array destructuring. The name of each variable matches the name of a property from the destructured object.

let myObject = { a: 1, b: 2, c: 3, d: 4};
let {b, d} = myObject;

console.log(b); // 2
console.log(d); // 4

And similar to arrays, you can also use ... to collect all the remaining properties, which were not assigned to a variable to a brand new object:

let myObject = { a: 1, b: 2, c: 3, d: 4};
let {b, d, ...remaining} = myObject;

console.log(b); // 2
console.log(d); // 4
console.log(remaining); // { a: 1, c: 3 }

Destructuring deep dive

For detailed explanation of destructuring please check the following article:

Destructuring objects and arrays in JavaScript

03 February, 2019

#Javascript

Destructuring allows you to assign items from arrays or properties of objects into single variables easily.

Conclusion

Three dots in JS can mean multiple things based on context.

You can use it as rest parameters in a function, so you are able to work easily with variable number of arguments. You can use a similar approach with array or object destructuring assignment, where the rest of the items, are nicely packed into an array.

The spread operator can be used to unpack an iterable into individual items, which can be used when creating objects, arrays or calling functions.

Java Raw String Literals

Mon, 26 Nov 2018 22:12:03 GMT

UPDATE: Raw string literals didn't make it to the Java 12 and will be postponed and reworked:

We're of course disappointed that this means it will take slightly longer for this feature to make it into the language, but we think that's the best choice.

While we can expect that for any language feature, there will be a nontrivial volume of "I would have preferred it differently" feedback, in reviewing the feedback we have received, I am no longer convinced that we've yet got to the right set of tradeoffs between complexity and expressiveness, or that we've explored enough of the design space to be confident that the current design is the best we can do. By withdrawing, we can continue to refine the design, explore more options, and aim for a preview that actually meets the requirements of the Preview Feature process (JEP 12).

String Literals

Currently (pre-Java 12), string literals in Java are in many cases unnecessarily hard to deal with and have some limitations.

First of all, Java does not support multi-line strings. If you want your string to span multiple lines, you have to concatenate multiple strings:

String myString = "This is my string" +
        " which I want to be " +
        "on multiple lines.";

It gets worse though. If you want your string to actually contain new lines, you need to insert \n after each line.

String myString = "This is my string\n" +
        " which I want to be \n" +
        "on multiple lines.";

Special characters such as newline are escaped with backslash \. This is necessary for double quotes " as they would terminate the string and also for backslash \ character as it would rather indicate escaped character. In addition to that there is a whole bunch of escape sequences in java:

Sequence	Usage
\t	Tab
\b	Backspace (remove previous character)
\n	New line
\r	Carriage return
\f	Form feed
\'	Single quote character
\"	Double quote character
\\	Backslash character

String literals are usually hard to read because they are plagued with escape sequences:

String myPath = "Open \"C:\\Program Files\\Java\\jdk1.8.0_151\"";

It gets especially bad with regular expressions, which can be using a lot of backslashes and are then hard to read because of escaping.

Raw Strings

Turns out, that other JVM languages already solved the problem with multiline strings and readability of escaped sequences. They are called Raw Strings. They can span multiple lines without concatenation and they don't use escaped sequences. You can use backslashes or double quotes directly. For example, in Kotlin, in addition to regular string literals, you can use Raw Strings with three double quotes """ instead of just one.

val text = """
This is my string
which I want to be
on multiple lines.
It can contain backslash \
Or even double-quotes "
"""

On JVM, also Groovy and Scala support raw strings using """. Java is like usually late to the party (similar to introducing JShell REPL) and only now, in version 12, introduces what is already common both in other JVM and non-JVM languages.

Raw String Literals in Java

In Java 12, one of the new features will be JEP 326: Raw String Literals. Unlike Kotlin, Groovy or Python, which use """ Java decided to use backtick ` notation, which is currently used by Javascript or Go.

String rawString = `This is my string
which I want to be
on multiple lines.
It can contain backslash \
Or even quotes "`;

But unlike javascript, backticks inside the string don't need to be escaped. Instead, you can use multiple backticks to start the string and then the same number of backticks to end it:

`This uses single backtick`
``This can contain backtick `, see?``
```Can use any number of backticks```

No matter how many backticks you use or whether you use raw or original string literals, the resulting .class file will be the same and both literals will produce java.lang.String.

Formatting

One problem with multiline raw strings is that they interpret all the characters on each line as part of the string. Imagine the following code.

public class Main {

    public static void main(String[] args) {
        String myString =
        `This is my string
        which I want to be
        on multiple lines.`;
        System.out.println(myString);
    }
}

Each of the lines of the raw string contains some spaces at left as indentation, so the code is nicely formatted - 4 spaces indentation because it is inside a class and another 4 because it is inside the main method. If you run it, you will see this:

This is my string
        which I want to be
        on multiple lines.

The string contains the indentation, which was supposed only to format the code to be more readable, but it is not intended to be part of the string.

A quick fix would be to remove the indentation, but it would be hard to read:

public class Main {

    public static void main(String[] args) {
        String myString =`This is my string
which I want to be
on multiple lines.`;
        System.out.println(myString);
    }
}

There are actually several new methods in the String class, which are useful for handling indentation.

The method align() trims horizontal and vertical indentation of the string and keeps just relative indentation of the lines.

public class Main {

    public static void main(String[] args) {
        String myString = `
        This is my string
        which I want to be
        on multiple lines.
            Note that after alignment,
                only relative indentation is preserved.
        `.align();
        System.out.println(myString);
    }
}

After alignment, the output will be this:

This is my string
which I want to be
on multiple lines.
    Note that after alignment,
        only relative indentation is preserved.

If you don't want to trim whitespace but only increase or decrease alignment, you can use String.indent(numberOfSpaces)

// Increase existing indentation of each line by 2
myString.indent(2);
// Decrease existing indentation of each line by 2
myString.indent(-2);

As a shorthand, if you call align() with an integer parameter instead of with zero parameters, it will align first and then indent by the given number:

// Align and then increase existing indentation of each line by 2
myString.align(2);

String Interpolation

Other languages, which currently support Raw String literals usually also support string interpolation. It is basically a fancy name for substituting variable placeholders in the string with values, such as:

    val name = "John"
    println("Hi, my name is ${name}.")

Unfortunately, this is still not possible in Java, not even Java 12. JEP 326 even explicitly states that this is not intended to be part of this proposal. However, it may be introduced by some other proposal in the future.

IntelliJ IDEA support

The good news is that since version 2018.3, IDEA already supports Raw String Literals.

You can convert from good old string literals to the raw variant and vice versa. IDEA will handle escape sequence conversion and splitting to multiline for you.

IDEA is also able to detect if you are unnecessarily using too many backticks and will be able to remove excess backticks for you.

In addition to this, it now supports actions such as spell-checking or regular expressions detection in Raw String Literals. You can read more in Preview Raw String Literals in IntelliJ IDEA 2018.3.

Try it yourself

As of 11/2018 Java 12 is not out yet, but you can already download early access build of JDK 12 and try Raw Strings for yourself.

Spring Boot Admin Tutorial

Sun, 18 Nov 2018 22:12:03 GMT

Monitor and manage you Spring Boot apps with a nice UI on top of Spring Boot Actuator endpoints.

Spring Boot Actuator

Actuator is a Spring Boot module, which adds REST/JMX endpoints to your application, so you can easily monitor and manage it in production. The endpoints offer health-check, metrics monitoring, access to logs, thread dumps, heap dumps, environmental info and more. I've covered Actuator in depth in the following article:

Actuator: Spring Boot Production Monitoring and Management

03 September, 2018

Monitor and manage your application in production with Spring Boot Actuator 2.x. Gather metrics or check health easily.

Spring Boot Admin

Actuator is powerful and great and it is easy and convenient to consume the endpoints with some other application - you just make a simple REST call. It is not so great when used by a human. For humans, it is much more convenient to have a nice user interface you can use to browse all the monitoring data and manage your application. This is actually what Spring Boot Admin Does. It provides you a nice UI layer on top of actuator endpoints with a few extra features on top.

Spring Boot Admin is not a core module provided by the Spring team, it was created by a company called Codecentric. Still, the code is publicly available on Github and it is free.

Client And Server

Unlike Actuator, Spring Boot Admin actually comes in two parts - Client and Server.

The Server part contains the Admin user interface and runs independently from the monitored applications. The Client part is in the monitored application, which registers with the Admin Server part.

This way, even if our application is down or not working properly, the monitoring server is still up and running. Now imagine you have multiple applications (such as Spring Boot microservices) and each of them can be running in multiple instances. With traditional Actuator monitoring, this is hard as you need to access each of them separately and you need to keep track of how many instances and where are running.

With Spring Boot Admin, each instance of your monitored application (Client) registers with the Server after it starts. Then you have a single point (Admin Server), where you can check them all.

Source Code

The source code with a finished application can be found in this Github repository.

Server Setup

Let's first look into how to setup Spring Boot Admin Server. Let's start with a fresh Spring Boot application. You can easily create one using Spring Initializr. Be sure to include the web module.

After creating the project, the first thing we need is to add the Spring Boot Admin Server dependency:

<dependency>
    <groupId>de.codecentric</groupId>
    <artifactId>spring-boot-admin-starter-server</artifactId>
    <version>2.1.0</version>
</dependency>

Note that even though the project is not created by Pivotal, you can find both the client and server modules for Spring Boot Admin in the Spring Initializr. Next, we need to enable Admin Server by annotating our main application class with @EnableAdminServer:

@SpringBootApplication
@EnableAdminServerpublic class SpringBootAdminServerApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringBootAdminServerApplication.class, args);
    }
}

And that's it. Now you can run your application and after opening http://localhost:8080/ you should see this:

The server is running, but no clients are registered yet. Let's change that.

Client Setup

Same as with server setup, the first step is to add a proper dependency to an existing application:

<dependency>
    <groupId>de.codecentric</groupId>
    <artifactId>spring-boot-admin-starter-client</artifactId>
    <version>2.1.0</version>
</dependency>

Then you need to define URL where is your Admin Server running. Add this line to your application.properties:

spring.boot.admin.client.url=http://localhost:8080

Adding Actuator

Now you should be able to run both the client and the server. Just make sure there is not a port conflict as both apps by default use 8080. For testing purposes, you can set server.port=0 in your application.properties, so your client will use a random port on startup. This way you can test launching multiple instances running on different ports.

When you open Admin Server UI, you should see your application. When you click the app name, a page with application details should show up.

If you see a screen like above with just a minimum of information, it means that you don't have Actuator as a part of your project. Remember, Spring Boot Admin uses Actuator endpoints under the hood. Fortunately, you need just to add a simple dependency and auto-configuration does take care of the rest.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

However, most of the endpoints are not exposed by Actuator by default. You need to change your configuration in application.properties to expose them:

management.endpoints.web.exposure.include=*

After exposing your Actuator endpoints, you should see much more information in your Admin interface:

For a detailed tutorial on Spring Boot Actuator configuration check this article.

Security

Now when everything is running, we should make sure our Actuator endpoints and the Admin UI are not publicly available to everyone.

Client Security

If you are already using Spring Security, the above will not work for you as the Actuator endpoints are secured by default and your Admin server will not be able to access them. If you don't use Spring Security yet, you need to add a dependency first:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>

For testing purposes, you can temporarily disable Actuator endpoint security by management.security.enabled=false. However, we do want to have security enabled. If you are using basic authentication, you can just provide username and password in your properties file. These credentials will be used by Admin Server to authenticate with the client's Actuator endpoints:

spring.boot.admin.client.instance.metadata.user.name=joe
spring.boot.admin.client.instance.metadata.user.password=my-secret-password

By default, if not configured otherwise, Spring Boot uses default user user and auto-generated password each time your app starts. You can check the password in the console during startup. If you want to provide username and password required by your app explicitly, you can define it in your properties:

spring.security.user.name=joe
spring.security.user.password=my-secret-password

Server Security

Same as with the client, we need to add Spring Security dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>

Now lets configure username and password required to login to the Admin Server in your application.properties:

spring.security.user.name=admin
spring.security.user.password=admin-password

Now in your Client you need to add these credentials as well, otherwise it will not be able to register with the server:

spring.boot.admin.client.username=admin
spring.boot.admin.client.password=admin-password

Now back to the Server part. The last thing we need is to add Spring Security configuration to protect the Admin user interface:

@Configuration
public class SecurityConfig extends WebSecurityConfigurerAdapter {

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        SavedRequestAwareAuthenticationSuccessHandler successHandler 
            = new SavedRequestAwareAuthenticationSuccessHandler();
        successHandler.setTargetUrlParameter("redirectTo");
        successHandler.setDefaultTargetUrl("/");

        http.authorizeRequests()
            .antMatchers("/assets/**").permitAll()
            .antMatchers("/login").permitAll()
            .anyRequest().authenticated().and()
            .formLogin().loginPage("/login")
            .successHandler(successHandler).and()
            .logout().logoutUrl("/logout").and()
            .httpBasic().and()
            .csrf()
            .csrfTokenRepository(CookieCsrfTokenRepository.withHttpOnlyFalse())
            .ignoringAntMatchers(
                "/instances",
                "/actuator/**"
             );
    }
}

What it does, is that it restricts the admin UI only to authenticated users using HTTP basic authentication and form login. The login page itself and static UI resources (javascript, HTML, CSS) are public. Otherwise, you could not log in. Then there is a cookie-based Cross-Site Request Forgery protection. You can see that some paths are ignored in CSRF protection - it's because the Admin Server currently lacks proper support.

Now after a restart, you should see a nice login screen protecting your Admin Server:

Cloud Discovery

Spring Boot Admin client is not the only way to register your applications with the server. Admin Server also supports Spring Cloud Service Discovery. You can read more in the offical documentation or in the Spring Cloud Discovery with Spring Boot Admin article.

Notifications

Once you have monitoring in place, you want to get notified when something goes wrong. The good news is that Spring Admin provides a wide variety of notification options.

If you visit the Admin Server page for the first time, it asks you for permission to display push notifications on your computer. Whenever there is an issue, you'll get a popup message.

Other notifications need a simple configuration. It usually means just providing a few entries in your application.properties. Currently supported services are:

Configuring Email Notifications

If you want to enable email notifications, you need to add Spring email dependency to your Server part:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-mail</artifactId>
</dependency>

Then you need to define your SMTP server, which will be used to send email notifications and credentials. Update your application.properties of your Admin Server.

spring.mail.host=smtp.foo.com
spring.mail.username=smtp-server-user
spring.mail.password=smtp-server-password

Then you need to define recipients and sender.

# Sender email address
spring.boot.admin.notify.mail.from="Spring Boot Admin <noreply@foo.com>"
# Comma-delimited list of recipient email addresses
spring.boot.admin.notify.mail.to=alice@foo.com,bob@bar.com
# Comma-delimited list of carbon copy recipient email addresses
spring.boot.admin.notify.mail.cc=joe@foo.com

Conclusion

Spring Boot Admin offers a nice and useful UI layer on top of Actuator Endpoints. What's more, it allows you to centrally monitor multiple applications with multiple instances, which is invaluable when working in the cloud and with microservices. Make sure though, that you sufficiently protect both your Client and Server. For further information, please check the official documentation.

Reactive Conf 2018: My volunteering experience

Fri, 09 Nov 2018 22:12:03 GMT

Reactive Conf 2018

Reactive Conf is a three-day conference focused on modern frontend and mobile development. I attended the last year for the first time and was impressed, so going this year again was a no-brainer, especially as it moved from Bratislava to Prague, my home city.

There were two stages with parallel talks. In addition to the main stage, there was also Discovery Stage, which was focused not only on presentations, but also on Ask Me Anything sessions with many of the speakers. This was a nice opportunity to ask questions, which in many cases, in my opinion, was even more interesting than the original talks. The questions were managed by sli.do - everyone could submit a question. Then people could vote for their favorite one. Questions with the most votes were then asked.

Each talk was roughly 30 minutes long including some time for questions. The talks were also publicly streamed, so even people not attending the conference could watch. Thirty minutes is a good length in the sense that you get to see more talks and it is easy to stay focused. On the other hand, speakers cannot really go into much detail. It is ok for many talks, but some of them could use more time.

Volunteering

Same as the last year, I didn't come as a regular attendee, but as a volunteer instead. The conference offers a number of "volunteer tickets", which you can apply for. The idea is simple - you get your ticket for free, but you help with running the conference. Of course, not for the whole time. Each day, you dedicate 1/3 of your time for working. The rest of the time, you are free to visit the talks, participate in workshops, attend the parties - anything you want.

The volunteer duties are diverse - registering visitors, helping speakers, dispensing gifts, navigating people around, moderation of questions, and much more. It does not really matter much that you don't get to see every talk. They are recorded and available after the conference.

However, the best part about participating as a volunteer is not about saving money. If it was the case, you could just wait for the youtube videos and watch them from the comfort of your sofa for free. Being there is an excellent opportunity to see the conference from the organizer's point of view. But most of all it is about meeting interesting like-minded people. Sure, you're supposed to network on a conference as an attendee, but I've always found it rather difficult with complete strangers as an introvert. For a volunteer, this is much easier and much more natural. You spend a lot of time with other volunteers from all over the world working together and get to talk to many people during your duties. You get a better chance to approach the speakers and access the VIP zone.

Day 1

The beginning of the conference was the busiest period for us volunteers. We had to register more than 1000 visitors. I spent most of the time providing various goodies to the attendees in exchange for their "Reactive Gift" vouchers.

After the talks finished, there was still a lot more to do. You could stay for a "Discover Czech" event, which was basically a party with a selection of classic Czech delicacies such as nakládaný hermelín (pickled cheese), utopenec (pickled sausages), various Czech pies or traditional plum spirit slivovice. Alternatively, you could join separate parties held by the conference's sponsors in their offices - STRV, MEWS, Kiwi or Socialbakers. For those not interested in partying, there was an opportunity to join a sightseeing tour of Prague.

Day 2

My duties changed on the second day. Now it was mostly about the Discovery Stage - showing the way, letting people in and so on. Then some basic cleanup after the conference.

After the talks, there was a closing ceremony. Then you could choose from a bunch of "Hangouts" with various companies such as Aragon, HackPrague or Green Fox Academy. Alternatively, you could continue in discovering Prague like the day before, which would end with a tasting of local wines, beers and spirits.

Day 3

The last day of the conference was different from the first two days. So-called "Festival day" was not at the same venue. Instead, it was hosted in the Prague Microsoft building. Unlike the first two days, this one was focused mainly on various workshops with limited capacity. It was necessary to register and pay a fee of 20 € in addition to the conference ticket. There were 14 workshops available in total ranging from 2 to 6 hours. Here is the list:

Introduction to Elm
GraphQL with React vs. Angular vs. Vue
Codelab: Build your first Flutter app
Introduction to ClojureScript
Vue.js
Evolving Patterns in React
Building type-safe applications with Flow and GraphQL
Reason DOJO: React and GraphQL
Reason DOJO: Cross-platform Games in Reprocessing
Simplify your data processing with BakeryJS
Introduction to E2E testing with Cypress.io
Service Workers at Your Service
How to build cross-platform desktop apps with React and Electron
Vuex like nobody is looking

In addition to workshops, there was one small stage. In the morning, there were just some recorded talks from the previous year. After that, there were regular talks for the rest of the day.

My duty for the last day of volunteering was moderating questions from attendees to speakers on sli.do for all the talks of the day. This was probably the best of my assignments as I got to see all the talks and it was interesting and kept me occupied.

After the talks and the final panel discussions with some of the speakers, there was an afterparty, still in the Microsoft building.

Impressions

Overall my impression of the conference was very good. Everything was professional and well organized. The event felt world-class, very international, which is not always the case in the Czech Republic. Compared to my last year's experience as a volunteer, everything was even smoother.

The selection of speakers was impressive with interesting and well-known presenters. Except for one questionable talk about "How I met Elon Musk", there was nothing bad.

While 30 minutes (or more like 25) per talk was sufficient in many cases, I was wondering whether it would be better to have some presentations longer. For some topics, it is just not enough time and the talk is more shallow that it could be. I liked that there were lightning talks as a short alternative to the regular presentations. Maybe it would also be good to introduce a longer alternative as well. Then again, some topics were covered in much detail during the workshops on the third day.

While the topics were interesting, I felt that there was too much focus on React neglecting other technologies, frameworks and subjects. At least in my opinion.

I really appreciated that there was a public live-stream and videos published after the conference, so I had a chance to watch the talks I missed because of my volunteering duties.

It was nice that there was a lot to do after the talks - Czech delicacies tasting, after-parties with various companies, discovering the city or Czech traditional live music.

Overall my experience was very positive, I'm sure I'll visit also the next year.

Talks

If you are interested in talks, which were at Reactive Conf, you can watch them all online.

Photos

You can check all the photos from the event or just watch the following 40-second video for a brief overview.

Conclusion

I had a great time volunteering at Reactive Conf 2018. It was really interesting to see a conference from the organizer's perspective and a great opportunity to meet many interesting people from all over the world - the organizers, the volunteers and the speakers. I definitely recommend this as an interesting alternative to just attending a conference.

Faster Development with Spring Boot DevTools

Fri, 02 Nov 2018 22:12:03 GMT

How to speed up your Spring Boot development even more with DevTools and make it more enjoyable and productive?

Setup

As usually with Spring Boot, the setup is really simple. All you need to do is to add the right dependency, and you are good to go. Spring Boot detects this and auto-configures DevTools accordingly.

If you are using Maven:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-devtools</artifactId>
    <optional>true</optional>
</dependency>

Alternatively, when using Gradle:

configurations {
    developmentOnly
    runtimeClasspath {
        extendsFrom developmentOnly
    }
}
dependencies {
    developmentOnly("org.springframework.boot:spring-boot-devtools")
}

Note that the dependency is declared as optional. This is important. This prevents DevTools dependency being transitively applied to other modules that depend on your project.

Automatic Restart

Whenever there is a change in files on your classpath, DevTools automatically restart your running application with the new changes applied. When developing locally, this can be valuable as you don't need to redeploy your application manually.

On its own, it wouldn't be so useful as restarts can still take a lot of time. Fortunately, these restarts are way faster than regular restarts because of a clever trick, which DevTools use.

You see, when developing an application, you usually change a class or a few and want to check results in your running application for feedback. You change a tiny fraction of your application as the majority of loaded classes are from frameworks and third party libs.

Under the hood, Spring DevTools use two classloaders - base and restart. Classes which do not change are loaded by the base classloader. Classes you are working with are loaded by restart classloader. Whenever a restart is triggered, restart classloader is discarded and recreated. This way restarting your application is much faster than usual and can be a viable alternative to dynamic class reloading with tools such as JRebel.

Triggering a restart in an IDE

The restart is triggered whenever there is a change on the classpath. However, this varies depending on your IDE. That means, it is not enough to just change your .java files. What matters is that your IDE actually updates .class files on the classpath.

When using IntelliJ IDEA, you need to build your project (Ctrl + F9 or Build → Build Project). You can also configure IDEA to rebuild automatically. Alternatively, you can open your Spring Boot run configuration and define what happens when you trigger an application update (Ctrl + F10):

In the first combo-box, you can select Update trigger file to trigger DevTools restart whenever you call the Update action. Alternatively, you can even select the option to try Hot Swap and restart using DevTools only if Hot Swap failed.

In the second combo-box, you can configure reloading all the static resources and templates when IDEA window loses focus (for example when switching to a browser window).

In Eclipse it is enough just to save your files.

Development only

The usage of the Spring Boot DevTools is intended only for development, not for production. If your application detects you're running in production, DevTools are automatically disabled.

For this purposes, whenever you run your app as a fully packaged artifact such as a jar with an embedded application server, it is considered to be a production app:

java -jar devtools-example-1.0.0.jar

Same applies when your app is started via special classloader, such as on an application server. In contrast, when you run an exploded artifact (such as in your IDE), your application is considered in development mode. The same applies when using spring-boot-plugin to run the application:

Maven:

mvn spring-boot:run

Gradle:

gradle bootRun

Live Reload

LiveReload is a useful tool, which allows you to instantly update your page in browser whenever you make changes in your files such as HTML, CSS, images and more. It even pre-processes files as needed - that means automatically compiling your SASS or LESS files.

Spring DevTools automatically launch a local instance of LiveReload server, which monitors your files. All you need to do is to install a browser extension, and you're good to go. It is not only useful for developing frontend of your application (in case you distribute it as a part of your Spring app artifact), but it can also be used to monitor and reload output of your REST API.

Properties override

When developing your application locally, you usually have different configuration needs than when running in production. One example can be caching. When in production, it is crucial to depend on various caches (such as templating engine's caches, caching headers for static resources and so on). In development, it can make you miserable by serving old data and not reflecting your latest changes. Another example may be enhanced logging, which can be useful in development but too detailed for production.

It is unnecessarily complicated to manage dual sets of configuration by yourself. The good news is that Spring Boot DevTools configure many properties for your local development out of the box.

spring.thymeleaf.cache=false
spring.freemarker.cache=false
spring.groovy.template.cache=false
spring.mustache.cache=false
server.servlet.session.persistent=true
spring.h2.console.enabled=true
spring.resources.cache.period=0
spring.resources.chain.cache=false
spring.template.provider.cache=false
spring.mvc.log-resolved-exception=true
server.servlet.jsp.init-parameters.development=true
spring.reactor.stacktrace-mode.enabled=true

You can check the list of all the properties in the DevToolsPropertyDefaultsPostProcessor.

Remote connection

In addition to local development, you can also connect to a remote application running DevTools. This is not intended for production environments as it can be a serious security risk. However, it can be very useful in pre-production environments.

Enabling remote connection

Remote connection is not enabled by default. You need to explicitly enable it by modifying your pom file:

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <configuration>
                <excludeDevtools>false</excludeDevtools>            </configuration>
        </plugin>
    </plugins>
</build>

Or with gradle, you need to set excludeDevtools = false:

bootWar {
    excludeDevtools = false
}

Then you need to set a secret password to be used for authentication when connecting to the remote application

spring.devtools.remote.secret=somesecret

Connecting to a remote app

Once your remote app is running, you can launch a remote connection session. Now all you need to do is to launch org.springframework.boot.devtools.RemoteSpringApplication with URL of your remote app as an argument. Note you should use https if possible.

Running the remote connection is easy in your IDE. In IDEA you need just to create a new run configuration. Go to Run → Edit Configurations... and create a new configuration with the + icon in the upper left corner. Choose Application type.

As the Main class select RemoteSpringApplication from the DevTools module and as a program argument pass URL of your remote app.

After you run this configuration, you should see a similar output if the connection to your remote app is successful.

  .   ____          _                                              __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _          ___               _      \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` |        | _ \___ _ __  ___| |_ ___ \ \ \ \
 \\/  ___)| |_)| | | | | || (_| []::::::[]   / -_) '  \/ _ \  _/ -_) ) ) ) )
  '  |____| .__|_| |_|_| |_\__, |        |_|_\___|_|_|_\___/\__\___|/ / / /
 =========|_|==============|___/===================================/_/_/_/
 :: Spring Boot Remote ::  (v2.0.6.RELEASE)

2018-11-02 17:24:42.126  INFO 16640 --- [           main] o.s.b.devtools.RemoteSpringApplication   : Starting RemoteSpringApplication v2.0.6.RELEASE on DESKTOP-6NJV4ON with PID 16640 (C:\Users\vojte\.m2\repository\org\springframework\boot\spring-boot-devtools\2.0.6.RELEASE\spring-boot-devtools-2.0.6.RELEASE.jar started by vojte in C:\projects\rest-docs-starter)
2018-11-02 17:24:42.130  INFO 16640 --- [           main] o.s.b.devtools.RemoteSpringApplication   : No active profile set, falling back to default profiles: default
2018-11-02 17:24:42.172  INFO 16640 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@3daa422a: startup date [Fri Nov 02 17:24:42 CET 2018]; root of context hierarchy
2018-11-02 17:24:42.679  WARN 16640 --- [           main] o.s.b.d.r.c.RemoteClientConfiguration    : The connection to http://localhost:8080 is insecure. You should use a URL starting with 'https://'.
2018-11-02 17:24:42.800  WARN 16640 --- [           main] o.s.b.d.a.OptionalLiveReloadServer       : Unable to start LiveReload server
2018-11-02 17:24:42.829  INFO 16640 --- [           main] o.s.b.devtools.RemoteSpringApplication   : Started RemoteSpringApplication in 1.212 seconds (JVM running for 1.877)

Once you connect to a remote app, DevTools monitors classpath changes same as it does for local development. However, instead of a local restart, it pushes the changes to the remote server and triggers restart there. This can be a lot faster than building the app and deploying to the remote machine.

Global configuration

You can configure DevTools using configuration properties as you would in any other Spring application. That usually means editing application.properties of your project. This configuration is separate for each application.

However, in some scenarios, it may be handy to have global configuration for ALL the applications running on the same machine. You can create a property file called .spring-boot-devtools.properties located in your $HOME directory. Config declared in this file is applied to all the applications running DevTools.

Limitations

Live Reload

Spring app using DevTools automatically launches a LiveReload server. Unfortunately, only one instance of this server can be running at the same time. To be more precise, just the first one will work. That applies not only to multiple instances of Spring apps with DevTools but to any other apps, which are also using LiverReload under the hood, such as Gatsby in development mode.

If you want to configure your Spring app not to launch a LiveReload server, you can do it in your application.properties:

spring.devtools.livereload.enabled=false

Shutdown Hook

DevTools are dependent on the shutdown hook of SpringApplication. It will not work correctly if you manually disabled the hook using:

springApplication.setRegisterShutdownHook(false);

By default, the hook is enabled, so you don't need to worry about it unless you explicitly disable it.

Collisions with third-party libraries

While DevTools should usually run properly, it may have conflicts with third party libs. In particular, there is a known issue with deserialization using standard ObjectInputStream.

In case of such conflict, you can disable the automatic restart by setting:

spring.devtools.restart.enabled=false

Restart will no longer be triggered. However, the restart classloader will still be used. If you need to disable the classloader completely, you need to do so before launching the app:

public static void main(String[] args) {
    System.setProperty("spring.devtools.restart.enabled", "false");
    SpringApplication.run(MyApp.class, args);
}

Even if you don't use th automatic restart you can still benefit from the other features DevTools provide.

Enabling lazy initialization

You can mark individual beans as lazily initialized using @Lazy annotation. This feature is in Spring for quite some time already. Since Spring Boot 2.2, you can toggle lazy initialization for all your beans using spring.main.lazy-initialization=true. This can be used on its own or combined with DevTools for even faster restart.

DevTools enables hot restart of your application in the same JVM. A significant benefit of hot restart is that it gives the JIT more of a chance to optimise the code involved in starting your application. After a few restarts, the original time of 2500ms is reduced by almost 80% to nearer 500ms. With lazy initialization, we can do even better. Setting spring.main.lazy-initialization sees our application restart in 400ms directly in the IDE.

Using lazy initialization for all your beans in a production application is questionable. It gives you excellent performance boost for start-up at the expense of longer first requests for individual beans. More importantly, your application does not fail fast anymore. Instead of crashing right when starting the app, it would fail only after directly requesting a misconfigured bean. This can be very dangerous as you would not discover many bugs until it is too late. Still, mass lazy initialization can be useful for speeding up development time as when working on a certain feature you usually work only on a fraction of your application and don't use the rest. An ideal compromise would be enabling mass lazy initialization for local development only (let's say using a spring profile) and disable it for deployed higher environments.

Conclusion

DevTools make your development of Spring Boot applications faster and easier by providing automatic restart and LiveReload functionality. I addition to this, it sets various properties to values more suitable for local development. Furthermore, it allows you to remotely connect to your application and still use most of its features. When running in production, DevTools are not used. For detailed information, see the official docs.

Actuator: Spring Boot Production Monitoring and Management

Mon, 03 Sep 2018 22:12:03 GMT

Basic build information

In one of my previous articles, I described how to obtain some basic information about the current build of your application at runtime in Spring Boot.

Detecting build version and time at runtime in Spring Boot

25 June, 2018

How to obtain artifact version, build time and other build information in a Spring Boot app at runtime?

While this can be handy, it is usually not sufficient. There is a lot more you are usually interested in. And most importantly, it does not involve just the information known at build time, but rather what is the application's current status at runtime. In these situations, you should use Spring Boot Actuator.

Spring Boot Actuator

In short, Spring Boot Actuator is one of the sub-projects od Spring Boot, which adds monitoring and management support for your applications running in production. It exposes various HTTP or JMX endpoints you can interact with.

This post covers Actuator 2.x. If you're using Spring Boot 1.x, be aware that there were many changes in the version 2 and you should look for version 1.x specific configuration instead.

Source code

The source code of example application using Spring Boot Actuator can be found here.

Adding Dependencies

The basic setup is really simple. You just need to add one dependency to your project - spring-boot-starter-actuator.

Maven:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Gradle:

compile("org.springframework.boot:spring-boot-starter-actuator")

As usual, once this dependency is on the classpath, Spring Boot is able to detect this and auto-configures Actuator for you. Just rebuild and start your application.

Up and Running

Now if you run your application, you can check all the available endpoints by accessing /actuator. That means, if you are running locally on port 8080, you'll need to open http://localhost:8080/actuator. You should see something similar to this:

{
  "_links": {
    "self": {
      "href": "http://localhost:8080/actuator",
      "templated": false
    },
    "health": {
      "href": "http://localhost:8080/actuator/health",
      "templated": false
    },
    "info": {
      "href": "http://localhost:8080/actuator/info",
      "templated": false
    }
  }
}

This is the list of all the available Actuator endpoints. As you can see, there is just two of them: health and info. Let's try /health first.

{"status":"UP"}

And now /info. It looks like it is returning just an empty json: {}.

More endpoints

Not very impressive so far. Of course, Actuator offers much more than this. Otherwise, it wouldn't be very useful. There are some examples of available endpoints, with the full list available in the official docs.

Endpoint	Description
health	Application health info
info	Info about the application
env	Properties from environment
metrics	Various metrics about the app
mappings	`@RequestMapping` Controller mappings
shutdown	Triggers application shutdown
httptrace	HTTP request/response log
loggers	Display and configure logger info
logfile	Contents of the log file
threaddump	Perform thread dump
heapdump	Obtain JVM heap dump
caches	Check available caches
integrationgraph	Graph of Spring Integration components

Exposing endpoints

For most of the endpoints, Actuator offers two ways of connecting to them:

HTTP REST endpoint
JMX endpoint

By default, as we already saw, only health and info endpoints are exposed over HTTP. However, all of them are exposed over JMX (where applicable) by default.

You can configure in your application.properties which endpoints should be exposed:

# Expose over JMX
management.endpoints.jmx.exposure.exclude=shutdown
management.endpoints.jmx.exposure.include=*
# Expose HTTP REST Endpoint
management.endpoints.web.exposure.exclude=
management.endpoints.web.exposure.include=health,info,metrics

As you can see, you can define endpoints by name or use * to represent all endpoints. You can combine include and exclude filters.

If you change your configuration to expose all the endpoints over HTTP and restart your app, your /actuator endpoint will now show many more URLs.

management.endpoints.web.exposure.include=*

Enabling / Disabling endpoints

You can configure not only whether an endpoint is exposed over HTTP or JMX, but also you can turn specific endpoints on/off. All the endpoints except shutdown are enabled by default (although not exposed over HTTP).

# Disable an endpoint
management.endpoint.[endpoint-name].enabled=false

# Specific example for 'health' endpoint
management.endpoint.health.enabled=false

# Instead of enabled by default, you can change to mode
# where endpoints need to be explicitly enabled
management.endpoints.enabled-by-default=false

/health

As we already saw, the information provided by /health endpoint is rather unimpressive.

{"status":"UP"}

Fortunately, this is just the default state and we can show much more details.

management.endpoint.health.show-details=always

Now the information available is much richer. The exact details vary based on your specific dependencies. For example, whether you use a data source and which one.

{
  "status": "UP",
  "details": {
    "db": {
      "status": "UP",
      "details": {
        "database": "H2",
        "hello": 1
      }
    },
    "diskSpace": {
      "status": "UP",
      "details": {
        "total": 497336446976,
        "free": 202989006848,
        "threshold": 10485760
      }
    }
  }
}

Of course, it may not be a good idea to show such information to everyone publicly. Fortunately, you can restrict access, so the details are available to users with specific roles only. Instead of always, you rather use when-authorized and then you can specify the allowed roles.

management.endpoint.health.show-details=when-authorized
management.endpoint.health.roles=ADMIN

/info

When we tried to call /info endpoint before, all we've got was just an empty response {}. Of course, we can do better. Let's examine various ways we can display more info.

Build Properties

In the article I referenced at the beginning of this post, I describe how to obtain information about the artifact and its build properties. The idea is simple, configure Spring Boot Maven/Gradle plugin to generate build-info.properties file, which contains the required information.

What's great is that if you do use Actuator, it automatically detects build-info.properties file and displays its contents through the /info endpoint. All you need to do is to add a simple config to your Spring Boot Maven/Gradle Plugin.

Maven pom.xml file:

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <executions>        <execution>            <id>build-info</id>            <goals>                <goal>build-info</goal>            </goals>        </execution>    </executions></plugin>

Gradle build.gradle file:

springBoot {
    buildInfo()
}

Info endpoint now provides build info information:

{
  "build": {
    "version": "1.0.0-SNAPSHOT",
    "artifact": "spring-boot-actuator-example",
    "name": "spring-boot-actuator-example",
    "group": "com.vojtechruzicka",
    "time": "2018-09-01T19:03:06.446Z"
  }
}

Git Properties

Actuator automatically detects git.properties file, which contains useful information about your git repository. To generate it, you'll need to add a specific plugin to your build config.

In Maven pom.xml:

<plugin>
    <groupId>pl.project13.maven</groupId>
    <artifactId>git-commit-id-plugin</artifactId>
</plugin>

Gradle uses a different plugin:

plugins {
    id "com.gorylenko.gradle-git-properties" version "1.5.1"
}

After rebuilding and restarting the /info endpoint displays some git info.

{
  "git": {
    "commit": {
      "time": "2018-09-01T19:46:31Z",
      "id": "493f071"
    },
    "branch": "master"
  }
}

Environment properties

Finally, Actuator automatically detects all the environmental properties, which start with info.. You can try it by adding a new property to your application.properties, which starts with info..

info.my-custom-property=What a nice property!

Then it is returned by the /info endpoint.

{"my-custom-property":"What a nice property!"}

What's useful is that it shows properties from various sources, not only application.properties.

Environment variables
Command line arguments
Servlet Config/Context params
And many more

/metrics

This endpoint exposes information about various metrics. By calling /metrics you get names of all which are available.

{
  "names": [
    "jvm.memory.max",
    "jvm.memory.used",
    "jdbc.connections.active",
    "http.server.requests",
    "jvm.gc.memory.promoted",
    "system.cpu.usage",
    "tomcat.cache.hit",
    "tomcat.cache.access",
    "jvm.gc.max.data.size",
    "jdbc.connections.max",
    "jdbc.connections.min",
    "jvm.memory.committed",
    "system.cpu.count",
    "logback.events",
    ...
  ]
}

If you want to display data for a specific metric, instead of calling /metrics you call /metrics/{metric-name}. For example, calling /metrics/jvm.memory.used will return response similar to this:

{
  "name": "jvm.memory.used",
  "measurements": [
    {
      "statistic": "VALUE",
      "value": 2.44780736E8
    }
  ],
  "availableTags": [
    {
      "tag": "area",
      "values": [
        "heap",
        "nonheap"
      ]
    },
    {
      "tag": "id",
      "values": [
        "Compressed Class Space",
        "PS Survivor Space",
        "PS Old Gen",
        "Metaspace",
        "PS Eden Space",
        "Code Cache"
      ]
    }
  ]
}

Metrics were completely reworked in version 2. Version 1 uses its own proprietary metric system, which is hierarchical. This does not work very well in the cloud with many application instances. Actuator version 2 uses an entirely new system for managing metrics. It is dimensional in nature and it utilizes Micrometer.

Micrometer provides a simple facade over the instrumentation clients for the most popular monitoring systems, allowing you to instrument your JVM-based application code without vendor lock-in. Think SLF4J, but for metrics.

So in addition to dimensional focus, you now have a nice facade to integrate various popular existing solutions such as Prometheus, Atlas, CloudWatch, Ganglia or New Relic. You can read more details in Micrometer: Spring Boot 2's new application metrics collector

Securing Actuator endpoints

Leaving your actuator endpoints exposed for anybody to access is not a good idea. That's why the vast majority of them are not exposed by default. Once you do expose them, you should make sure they are adequately protected.

The good news is that if you use Spring Security, Actuator endpoints are secured by default. If you're not using Spring Security, you can add this dependency:

Maven:
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>

Gradle:
compile 'org.springframework.boot:spring-boot-starter-security'

If you rebuild and restart your app, you'll notice that now you are required to log in if accessing the actuator endpoints. By default, the username is user and the password is randomly generated and printed to the console every time the application starts:

Using generated security password: f7b833aa-5a1c-42f4-b913-70c1abe47cb6

Protecting all the endpoints like this may be sufficient in many cases, you'll just usually change how user's credentials are validated. If you need more fine-grained control, you should create a configuration class extending WebSecurityConfigurerAdapter and override the configure method:

@Configuration
public class ActuatorSecurityConfiguration extends WebSecurityConfigurerAdapter {

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.authorizeRequests()
            .requestMatchers(EndpointRequest.to(ShutdownEndpoint.class))
                .hasRole("ADMIN")
            .requestMatchers(EndpointRequest.to(HealthEndpoint.class, InfoEndpoint.class))
                .permitAll()
            .requestMatchers(EndpointRequest.toAnyEndpoint())
                .fullyAuthenticated()
            .and().httpBasic();
    }
}

In the example above, the /shutdown endpoint is accessible only for an authenticated user with role ADMIN. Endpoints /health and /info are accessible for anybody. All the other endpoints are accessible only for fully authenticated users.

Notice that we are not matching by URL of the endpoints but rather by class, so we are not tightly coupled to the URL to which an endpoint is mapped to. If URLs are reconfigured, we don't need to change our security config.

Actuator 1.x vs 2.x

Many of the changes brought with Spring Boot 2.0 are in fact in the Actuator module.

Actuator endpoints are now technology independent. Before, they relied on Spring MVC. Now it does not matter whether you use MVC, Jersey or the new Spring WebFlux.

In version 1.x there was a special security configuration just for actuator endpoints. This was dropped and you now use Spring Security directly as you would for any other endpoint. It is much simpler and more consistent.

The old version of Actuator used to have proprietary metrics, but 2.x uses Micrometer. This was later backported even to 1.x.

There were some new endpoints introduced, some were renamed to more descriptive names. The configuration properties were also reworked and are now neatly grouped together by common prefix.

More detailed description of the changes can be found in Spring Boot 2.0 actuator change analysis. If you are migrating from version 1.x this guide may come in handy.

Adding Graphical User Interface

Actuator endpoints are great, but monitoring and managing your application just through JMX and HTTP endpoints may be too low-level and cumbersome for many. If you prefer a nice GUI instead of interacting with endpoints directly, there's a great tool just for this. It is called Spring Boot Admin. It is a third-party open-source project, which gives you a nice UI to manage and monitor your applications. What's more, a single instance of Spring Boot Admin can monitor multiple applications and/or multiple instances of each application. That's really useful in the cloud environment with many dynamic instances of your app.

Spring Boot Admin Tutorial

18 November, 2018

Monitor and manage you Spring Boot apps with a nice UI on top of Spring Boot Actuator endpoints.

Conclusion

Spring Boot Actuator offers a powerful solution for monitoring and managing your application in production. It offers interaction either over JMX or HTTP endpoints. If you prefer GUI instead try Spring Boot Admin on top of Actuator Endpoints. However, bear in mind that having public endpoints which allow you to tinker with your app and expose sensitive data is not a good idea. Always be sure to secure your endpoints.

Staticman: User generated content made static

Sat, 25 Aug 2018 12:12:03 GMT

Dynamic content in static sites

Static JAMStack sites are great. If you are not familiar with them, you can check my older post on how I migrated my blog from WordPress to a static site using Gatsby.

Migration to GatsbyJS and JAM stack from WordPress

25 March, 2018

How and Why I migrated from WordPress to static JAM Stack site built with Gatsby JS.

Long story short, you get excellent performance, increased security and more. Static sites do have their limitations though. Most of the pages use some kind of user-generated content. Usually, it is comments, user reviews, discussions, and various others.

With traditional static pages, this is somewhat problematic. With users submitting new content such as comments all the time, your site cannot be truly static. You usually end up with a solution such as Disqus. That is - a third-party service, which collects, manages and displays the user-generated content on your site. You have your core content static and then load a bunch of javascript, which fetches the dynamic content from third-party API. That's what A in JAMStack actually stands for - API.

This may be necessary for dynamic and ever-changing data such as stock prices, weather forecast and so on. However, comments or user reviews actually don't belong to this category. They are, in fact, static. Once you post a comment to a discussion it hardly ever changes. The discussion thread stays exactly the same until someone new adds a comment.

What if you could build your static site after a user comments including their new submission? Instead of loading the comment thread dynamically after the page loads, the whole thread could be core part of your static page. This way your site would be truly static with all the advantages.

Staticman

It turns out you can achieve this easily with a tool called Staticman. The idea is simple.

Provide a submission form to your static site. For example a form for submitting a new comment.
Since static sites have no backend, the form submits to a Staticman service instead.
Staticman pushes the changes to your git repo.
As usual, your site can be automatically rebuilt and redeployed on code change.
The user submitted data is now a core part of your static site.

How great is that? Of course, you don't want users spamming your site, so Staticman offers you an option to moderate submissions by creating a Pull Request instead of directly pushing to your branch. If you want more sophisticated protection, it does offer integration with Akismet anti-spam filter or re-captcha.

Setup

Granting Github access

The first thing you need to do is to grant Staticman privileges to access your Github repo. Go to Settings → Collaborators section of your repository and add staticmanapp as a collaborator.

After adding Staticman as a collaborator, you should see a pending invite:

Now you need to open the following URL in your browser to make Staticman accept your invite:

https://api.staticman.net/v2/connect/{your GitHub username}/{your repository name}

With a real data it looks something like this:

https://api.staticman.net/v2/connect/vojtechruz/staticman-example

If all goes well, you should receive response OK!. Now, in the collaborators section, you should no longer see the pending invite.

Adding a configuration file

Staticman loads its configuration from a yaml file called staticman.yml. A sample file can be found here. A detailed explanation of all the available configuration options can be found in the official documentation.

Here is a list of some of the most important configuration properties:

Property	Description
allowedFields	An array of the form fields, which should be accepted. If a field is received and it is not specified here, it is discarded.
branch	Name of the git branch Staticman should use.
commitMessage	Commit message to be used.
fileName	Each entry is stored as a file. This property determines the name. You can use placeholders such as @{id] or {@timestamp}.
format	Format of the files created. Can be yaml, json or frontmatter
moderation	If true, Staticman will create a Pull Request instead of pushing directly to your configured branch.
pah	Path to a directory where your Staticman files will be created.
requiredFields	Array of required fields. If some of them are missing, the entry will be rejected.

Creating a form

Now all you need is to create a form, which accepts user input and submits it to Staticman. You need to define all the fields which you want to collect and submit the form to:

https://api.staticman.net/v2/entry/{GITHUB USERNAME}/{GITHUB REPOSITORY}/{BRANCH}/{PROPERTY}

An example of such form:

<form method="POST" action="https://api.staticman.net/v2/entry/vojtechruz/staticman-example/master/comments">
  <label><input name="fields[name]" type="text" />Name</label>
  <label><input name="fields[email]" type="email" />E-mail</label>
  <label><textarea name="fields[message]"></textarea>Message</label>
  
  <button type="submit">Submit</button>
</form>

Note the /comments section at the end of action URL. This is a section identifier (marked as {PROPERTY} in the url template above), which allows you to have multiple configurations for your site. Eg. One for comments, one for user reviews etc. In this case, we're saying that we want to use configuration specified under the comments section of your configuration file.

comments:
  allowedFields: ["name", "email", "message"]
  ...
reviews:
  allowedFields: ["name", "email", "rating"]
  ...

If you want to redirect the user to a specific url after submit you can do it by including this hidden field:

<input name="options[redirect]" type="hidden" value="https://www.example.com" />

Integrating with your static site generator

Each user submission is stored as a separate file. You can define the location and format of these files in your configuration file.

Each file contains fields submitted by the user plus any automatically generated fields, which you can define in your config file.

_id: 749a3b20-a7a4-11e8-9d49-9fff9455a2a1
name: Vojtech Ruzicka
email: vojtech.ruz@gmail.com
message: Hello StaticMan!
date: 1535118541

What you need to do is to make sure your static site generator reads this data (you can set a different format than yaml if you need to) and generates static content based on it. This will, of course, vary greatly depending on which generator you use.

For example, if using Gatsby, you can use gatsby-transformer-yaml to read the yaml data and then just populate your comments section based on this data as you would with any other data source.

Moderation

You can turn on moderation of user submissions by setting moderation: true in your config file. Staticman then creates a Pull Request for each user submission, which you can either accept or decline.

Advanced features

Running your own instance

If using Staticman as a service does not suit your needs for any reason, you can run your own instance.

Akismet

Staticman offers integration with Akismet spam filter. This can be great when running your own instance, but unfortunately does not work currently on public Staticman instance as it is using shared Akismet account under the hood, which reached the maximum usage limit.

Email notifications

Staticman supports email notifications on new user submissions using Mailgun. You just need to enable the notifications in your config file and provide your Mailgun domain and API key.

Re-captcha

Staticman supports re-captcha integration. You'll need to create a re-captcha account first. You will receive a site-key and secret. Your secret is not used directly by Staticman and you need to encrypt it first.

https://api.staticman.net/v2/encrypt/SECRET

You should receive encrypted secret back in the response. You then use it in your staticman.yml config file:

reCaptcha:
  enabled: true
  siteKey: "SITE-KEY"
  secret: "ENCRYPTED-SECRET"

Then you need to include re-captcha javascript in your site:

<script src='https://www.google.com/recaptcha/api.js'></script>

Then the element where it should be rendered:

<div class="g-recaptcha" data-sitekey="YOUR-SITE-KEY"></div>

Finally you need to include these hidden fields in your Staticman form:

<input type="hidden" name="options[reCaptcha][siteKey]" value="SITE-KEY" />
<input type="hidden" name="options[reCaptcha][secret]" value="ENCRYPTED-SECRET" />

You can find detailed info about re-captcha integration here.

Example

The source code of a simple GatsbyJS site using Staticman can be found here.

Running example is available here.

Conclusion

Staticman is an excellent tool if you want to have user-generated content on your site while remaining truly static. You no longer need to rely on third-party services, to manage and store your content. There is no need to load the user-generated content from third-party API at runtime. This should give you better performance and you have direct control over your data. If you need moderation of submitted content, Staticman can create Pull Requests, so you can decide which submissions are acceptable and which not.

Detecting build version and time at runtime in Spring Boot

Mon, 25 Jun 2018 22:12:03 GMT

This post covers obtaining simple build-related information without adding any additional dependencies. For much more detailed info, various metrics and health monitoring you should consider using Spring Boot Actuator:

Actuator: Spring Boot Production Monitoring and Management

03 September, 2018

Monitor and manage your application in production with Spring Boot Actuator 2.x. Gather metrics or check health easily.

Obtaining build information

It can often be useful to obtain information about artifact, version, build time and other at runtime. Sure, most of this information is already in your pom.xml file, but it can be tricky to retrieve these when the application is running.

Having such information at runtime can be useful. For example, imagine a scenario, where you expose a REST endpoint, which can tell the client what your current version of the application is, when was it built and so on. It can be useful because you can quickly determine what version of the app is currently deployed. This can be especially important in non-production environments, where the app is frequently deployed or even with continuous deployment in production. In such cases, it is vital to know what build exactly is currently running when testing and submitting bug reports. Maybe the issue reported is already fixed in a newer version or maybe the bug still occurs because the new version is implemented, but not deployed yet.

In any case, having build information can be handy and it is useful to know how to obtain it at runtime. In Spring Boot, it is fortunately quite easy.

Build plugin configuration

If you are using Spring Boot, your pom.xml should already contain spring-boot-maven-plugin. You just need to add the following configuration.

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <executions>        <execution>            <id>build-info</id>            <goals>                <goal>build-info</goal>            </goals>        </execution>    </executions></plugin>

It instructs the plugin to execute also build-info goal, which is not run by default. This generates build meta-data about your application, which includes artifact version, build time and more.

If you are using Gradle, just add the following to your build.gradle file:

springBoot {
    buildInfo()
}

Accessing Build Properties

After configuring your spring-boot-maven-plugin and building your application, you can access information about your application's build through BuildProperties object. Let the Spring inject it for you:

@Autowired
BuildProperties buildProperties;

Now you can access various information from this object.

// Artifact's name from the pom.xml file
buildProperties.getName();
// Artifact version
buildProperties.getVersion();
// Date and Time of the build
buildProperties.getTime();
// Artifact ID from the pom file
buildProperties.getArtifact();
// Group ID from the pom file
buildProperties.getGroup();

Adding custom properties

If predefined properties are not enough, you can pass your own properties from pom.xml file to BuildProperties.

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <executions>
        <execution>
            <goals>
                <goal>build-info</goal>
            </goals>
            <configuration>                <additionalProperties>                    <java.version>${java.version}</java.version>                    <some.custom.property>some value</some.custom.property>                </additionalProperties>            </configuration>        </execution>
    </executions>
</plugin>

You can pass a value directly or use your custom properties defined in the <properties> section of your pom.xml and then referenced using ${property.name} placeholder.

You can access custom properties defined this way by calling buildProperties.get("property.name").

For Gradle projects, custom properties can be defined this way:

springBoot {
    buildInfo {
        properties {
            additional = [
                'property.name': 'property value',
                'other.property': 'different.value'
            ]
        }
    }
}

How it works under the hood

When build-info of spring-boot-maven-plugin is run, it generates a property file containing all the build information. By default, it is located at ${project.build.outputDirectory}/META-INF/build-info.properties, but you can customize it by providing outputFile parameter. The file looks something like this:

#Properties
#Sat Jun 23 15:58:56 CEST 2018
build.version=0.0.1-SNAPSHOT
build.group=com.vojtechruzicka
build.name=spring-rest-docs-example
build.artifact=spring-rest-docs-example
build.time=2018-06-23T13\:58\:56.742472800Z

When Spring detects there is this file on the classpath, it creates BuildProperties bean unless it is explicitly declared. This is configured in org.springframework.boot.autoconfigure.info.ProjectInfoAutoConfiguration. This is a nice example of Spring Boot Auto-Configuration, where certain beans can be created just by having specific files on the classpath.

@ConditionalOnResource(resources = 
"${spring.info.build.location:classpath:META-INF/build-info.properties}")
@ConditionalOnMissingBean
@Bean
public BuildProperties buildProperties() throws Exception {
    return new BuildProperties(
            loadFrom(this.properties.getBuild().getLocation(), 
            "build"));
}

Detecting Spring profiles

It is no doubt useful to know which version of your artifact is deployed and when it was built. However, it is usually not enough. Often Spring applications use various profiles, which can significantly change the behavior. Typical usage is, for example, having a separate profile for each environment (DEV, UAT, PROD, ...). Depending on the profile, the correct environmental configuration can be loaded such as DB connection and more.

It is useful to be able to determine current profiles as sometimes the app can be run with different profiles than expected. To detect the current profiles, you need just to inject Environment object and then you can simply obtain them by calling getActiveProfiles().

@Autowired
private Environment environment;

environment.getActiveProfiles();

What's more, since you already have environment object, you can obtain any environmental properties by calling environment.getProperty("property.name").

Spring Actuator & Admin

While this approach gives you basic build and version info, sometimes you may need a more powerful tool. Spring Boot Actuator is a sub-project of Spring Boot, which adds some production-grade monitoring and management tools exposed as REST and JMX endpoints. In fact, it can easily be configured to use build information provided by BuildProperties and provide them through one of its endpoints.

Spring Boot Admin is a community project, which provides a nice user interface on top of Spring Actuator endpoints, so the app is more comfortable to manage through a nice admin GUI.

Conclusion

Having access to version and build information at runtime can be quite useful. In Spring boot application, you can easily obtain the info by altering the Spring Boot Maven/Gradle plugin configuration to generate the build.properties file and then accessing it through BuildProperties object.

For simple scenarios, this is an easy and quick solution and should work for you. If you need something more powerful, look at Spring Actuator or Spring Admin, which can provide the build metadata functionality plus a lot more.

Easy git hooks with Husky

Thu, 21 Jun 2018 22:12:03 GMT

In this article, you will learn how to easily create and manage git hooks for your Node/NPM projects using Husky.

Git Hooks

Simply put, git hooks are custom scripts, which can be run automatically when specific events occur. There are client-side hooks which are triggered on actions such as committing or merging. Server-side hooks run in situations such as receiving push data from the client.

The hooks can perform any custom logic and most importantly reject the action performed if something is not in order. For example, you can abort commit if its message does not contain issue tracker's issue ID. Or, you can reject it if static code analysis fails. It can be pretty useful if you want to make sure your codebase stays clean or you want to enforce certain quality policies.

But how can you actually install and manage these hooks? Whenever you clone a git repository, all the git's data for your project is stored in .git directory inside your folder. It contains several files and sub-directories, one of which is called hooks. Inside you'll find a bunch of files.

Each of them is a script, which is executed once a specific event occurs. The name of the event matches name of the file. So for example pre-commit is executed before you commit your changes.

As you can see, all the files have extension .sample. Git ignores these files unless you rename them - you need to remove the .sample extension to enable these hooks. Inside the sample files, you can find some description and sample implementation, which you can use as a starting point when implementing your own hooks.

What is it good for?

Now, let's look at some specific examples of what can be achieved with git hooks. Since they're just scripts, you can do pretty much anything. Usually, that means making various quality checks.

You can make sure user has their name and email filled. You can check the commit message is properly formatted. You can try to build your app and reject the commit if the build fails. You can run tests to make sure they are passing before commit.

Typical usage is also static code analysis or linting. That means checking your code for common issues, bad practices, naming conventions and more. It can also be useful to run a tool such as Prettier to make sure the code is nicely formatted before commit. This saves many headaches during code reviews. You can even check your code for security vulnerabilities with a tool like Snyk.

Snyk – Detecting dependencies with known vulnerabilities

22 November, 2017

#Security

How to detect and fix security vulnerabilities in your dependencies using Snyk?

Distribution to team members

With server-side hooks the distribution is easy. You usually have just one primary server repository. This means all the team members usually push their changes to one central place. You set up your hooks there and you're done.

With client-side hooks, this gets more complicated. When you clone a repository, the hooks are not transferred to the client. That means a freshly cloned repo does not have any hooks at all no matter what hooks you have server-side. If you want your team members to have unified set of git hooks, you need to distribute them somehow and make sure they are included in their git hooks sub-directory.

The most basic solution is to have some shared location, where you store your hooks and then tell your developers to download them and put them in their hooks directory. Of course, you cannot be sure that they will actually do it. The problem is they need to know they should do this and how. And even if they do, they may be lazy or just ignore your policy.

You can improve this basic solution a bit by having hooks in your project's repo and just let your developers run a custom script like this one, which copies them over to their hooks directory. Alternatively, git offers an option to change the destination of the hooks directory to a custom location:

git config core.hooksPath YOUR_DIRECTORY

These solutions make the distribution easier, but don't solve the core issues.

Husky

Installation

You can install Husky simply by running:

npm install husky --save-dev

Alternatively, with yarn:

yarn add husky --dev

Adding hooks

Adding hooks with Husky is easy. You just need to edit your package.json. There you define which scripts should be run on which git event.

For Husky 1.0.0+ use:

{
  "husky": {
    "hooks": {
      "pre-commit": "npm test",
      "pre-push": "npm test",
      "...": "..."
    }
  }
}

If you are using a Husky version older than 1.0.0 (which is in Release Candidate as of 6/2018), the syntax is slightly different. You add your Husky hooks directly in your scripts section.

{
  "scripts": {
    "precommit": "npm test",
    "prepush": "npm test",
    "...": "..."
  }
}

The example above runs all your tests before you commit and before you push and if your tests fail the git action is not executed. Of course, you can run any other npm script rather than npm test.

Autofix issues before commit

It is useful to break the build if something is not right, but it is even more useful to auto-correct issues before committing. For example, you can prettify your code using Prettier before commit or you can auto-fix lint issues, which can be resolved automatically. It is much easier this way.

Fortunately, there is a tool exactly for this. It is called lint-staged. You can install it by:

npm install --save-dev lint-staged

Now on your pre-commit action, you execute lint-staged instead of your linter directly. In lint-staged configuration you define what should be executed:

{
"husky": {
    "hooks": {
      "pre-commit": "lint-staged"
    }
  },
 "lint-staged": {
   "*.js": ["eslint --fix", "git add"]
 }
}

When you try to commit now, lint-staged can change your files before the commit is executed. However, what's really cool is that you are linting only files staged to commit, not your whole project. That means the whole process is way faster.

JetBrains IDE bug

UPDATE: The bug described below is fixed in version 2018.3, if you are using an older version you can use this workaround:

"post-commit": "git update-index -g"

The good news is that git hooks configured in this way are executed not only when you use git from your command line, but also from an IDE. The bad news is that JetBrains IDEs (IDEA, Webstorm, ...) have currently a nasty bug (see IDEA-135454) and doesn't work well with this setup.

~~The issue is not resolved for several years, but fortunately, there is a workaround. You just need to add this post-commit hook:~~

~~"post-commit": "git update-index -g"~~

~~Of course, this is just workaround until the issue is fixed. JetBrains's issue tracker contains voting functionality, so be sure to vote for this issue to be resolved if it causes you trouble.~~

Continuous integration

One thing to note is that Husky installs the hooks only when not running on a Continuous Integration server. Husky can detect it is running as a part of CI job and will not install any hooks.

Ignoring

Client-side hooks can be useful, but you cannot rely on them too much. They are just the first level of defense. You cannot be 100% sure that they get executed. They can be ignored on demand by adding a command-line option:

git commit --no-verify

To make things even easier, the hooks can be disabled using certain environmental variables. Because of this, it is still useful to enforce the same functionality on the server.

Performance

While client-side hooks such as pre-commit can be very useful, you need to keep in mind that they take some time to execute. Commits, which are usually very fast as they happen only on the client, can suddenly take very long time. You may be tempted to run all the tests, static code analysis, prettification and more before each commit. When a commit takes ages, your developers will not be happy and may be tempted to ignore the hooks when performing their git commands. So you should find the right balance between what needs to be performed on the client and what can be a server-side hook.

Conclusion

Husky is a useful tool, which allows effortless creation and management of git hooks on the client. You no longer need to distribute your hooks manually. As with everything, keep the number of your client-side hooks in moderation to avoid long execution times.

Adding search to your static JAMStack site

Sun, 17 Jun 2018 22:12:03 GMT

Searching static sites

A few months ago I finally managed to migrate my blog from WordPress to a static site build with GatsbyJS and deployed on Netlify.

Migration to GatsbyJS and JAM stack from WordPress

25 March, 2018

How and Why I migrated from WordPress to static JAM Stack site built with Gatsby JS.

I was very impressed with the results of the migration, especially performance-wise. I was able to migrate all the functionality, except site-wide search.

With server generating your pages, searching is easy. WordPress offers search option out of the box. With static sites, which have no dynamic backend, this gets trickier.

Client-side search

One option to enable search for your application is to use full client-side search with libraries such as js-search. The idea is simple. When you build your static site using your favorite static site generator, you also generate a search index of your pages. That basically means generating a lot of JSON with the contents of your site and then distributing this along with your static files. The search is lightning-fast because everything happens on the client. The problem is that this solution may not be good for full-text search. Bringing this much data to the client just for search can mean a significant performance hit. Especially if your site has a lot of content. Moreover, just a fraction of users is likely to use the search option. The rest just download useless junk. You sacrifice one of the greatest advantages of going static - performance just for adding a search option.

Good old Google Custom Search

All right, so for full-text search, it may not be such a good idea to download the whole search index to the client. Since our static site does not have a dedicated backend, we need to use third-party to handle the search. The first obvious choice is to use Google search. Google Custom Search to be precise.

The advantage is it doesn't need to download the whole search index. You use Google to search your site only. There are some disadvantages though. First one is that Google injects ads to your search results. The second one is that even though you can customize the look and feel of the search results page to some degree, it is still obvious that you utilize Google. The results page will likely not fit very well the look and feel of your page and arguably it does not look very well integrated and professional. Especially with the ads. Like you didn't care enough to provide custom search as other sites do.

Alternatively, if you don't mind doing some extra work, custom search offers an API, so you can retrieve results programmatically and seamlessly integrate them in your page. This, however, somewhat negates the most significant advantage of Google Custom search, which is its easy setup.

If you don't care about the downsides much, this can be very easy and quick way to add search for your static site. If you don't like it, there are still alternatives.

Algolia DocSearch

Algolia DocSearch is similar to Google Custom Search in a way that you don't need to download the whole search index to the client and that the search is provided by third-party service.

There are some advantages over Google Search though. First of all, no ads. Also, everything integrates with the look and feel of your site nicely as you just mark a text input on your page as a search box to be handled by DocSearch. It then attaches autocomplete functionality to this search box. Whenever you start typing you get results from DocSearch. Directly in your page with possibility to easily override default styles. So the setup is really easy and your pages are indexed for you automatically.

Requesting access

The first step you need to make is to request DocSearch for your site. Just fill a simple form on their website.

If your request gets approved, you're good to go. They will configure the crawler for your site, which will gather search data once a day. And they will send you JS and CSS to include on your page to enable the search. That's it.

But your request may get rejected (as in my case) because DocSearch is primarily intended as a solution for searching documentation pages of open source projects.

However, after the initial rejection, I did ask nicely whether they can provide the service for my blog. They were really kind to me and allowed me to register.

DocSearch focuses on documentation content that helps the community. It looks like your content is helpful this why we will provide you DocSearch.

DocSearch on your own infrastructure

However, even if your request is rejected, nothing is lost. The whole DocSearch is open-source and you can install and run it on your own infrastructure. It's a viable alternative as the search metadata is still managed and stored externally and collected by DocSearch crawler.

Setup and configuration

One of the advantages of DocSearch how easy it is to set up and configure. You just need to add the following to your page (Algolia team provides you with exact values and details after the registration):

<!-- at the end of the HEAD --> 
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.css" /> 

<!-- at the end of the BODY --> 
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.js"></script> 
<script type="text/javascript"> docsearch({ 
 apiKey: '### PROVIDED  BY ALGOLIA ####', 
 indexName: '### PROVIDED  BY ALGOLIA ####', 
 inputSelector: '### REPLACE ME ####', 
 debug: false // Set debug to true if you want to inspect the dropdown 
}); 
</script>

You just need to provide proper selector (such as CSS class or id) for the input search element. Algolia team also configures the project for you and they store the configuration file in their git repo. In case you need to make any changes, you just create a Pull Request, which gets quickly merged. In the config file, you define a hierarchy of your pages sections. That means, for example, that H1 headings have more weight in search results than H2. And H3 have even less weight. Ordinary paragraphs have the least relevance. With custom selectors you can define where precisely in your page is your content, so you can easily ignore sections such as footer or navigation. Here is an example of a simple configuration for my site:

{
  "index_name": "vojtechruzicka",
  "start_urls": [
    "https://www.vojtechruzicka.com"
  ],
  "stop_urls": [],
  "selectors": {
    "lvl0": {
      "selector": "",
      "global": true,
        "default_value": "Search Results"
    },
    "lvl1": ".docSearch-content h1",
    "lvl2": ".docSearch-content h2",
    "lvl3": ".docSearch-content h3",
    "lvl4": ".docSearch-content h4",
    "text": ".docSearch-content p, .docSearch-content li"
  },
  "conversation_id": [
    "593663647"
  ],
  "nb_hits": 1890
}

If you're looking for a more complex example, you can check configuration of GatsbyJs.

Algolia Search

DocSearch is primarily focused on documentation and full-text search. However sometimes you need something more powerful and sophisticated. In such case, Algolia's flagship product - Algolia Search is better for you.

Instead of working with the full text of your pages, you can define your own structured data in the form of JSON objects and Algolia works with them. Each of these objects can have many properties and you can search across all of them.

On top of this Algolia offers many advanced features, which are not available in DocSearch. You can define how the search results should be ranked, so the user receives the most relevant results. You can define synonyms for search terms. You can provide correction alternatives to common typos. There is a nice dashboard and search logs. You can even have analytics with a paid plan.

Uploading search data

Algolia does not crawl your content automatically as DocSearch does. You need to upload your data to their servers using REST API. They offer clients for a number of languages and frameworks such as Java, C#, PHP, Python, Django, Laravel, Rails or Symfony.

While this may be useful for traditional applications, it is not of much use for static sites. In this case, you need to build your JSON objects as a part of building your static site and then upload them using their REST API.

Of course, you can do it manually and it will work. With some static site generators, it is your only option. Fortunately, some of the generators provide plugins for Algolia, which make the process much easier. For example there is a plugin for Jekyll or GatsbyJs (as of 6/2018 in beta).

Search your videos

If you also work with videos, Algolia offers another exciting service. It is called Algolia TalkSeach. It allows you to add search to your youtube videos. It fetches transcript for the video and makes it searchable, so you can easily navigate to the part of the video you are interested in.

It is intended primarily for conferences and meetups, but again, they may be kind enough to make an exception for you.

Conclusion

When adding search to static sites, you need a different approach than in traditional apps where your backend can handle the search for you. Adding Google Search can be quick and painless, but the customization is limited and it does not feel like a nice integrated solution.

Having client-side search offers a fast solution, which can be good if you want your app to work offline as a Progressive Web App. Of course, you need to download the whole search index to the client, which can be too much once you have a lot of content.

Algolia DocSearch is an easy to set up alternative, which crawls your site for you. So the data is stored on the server, so you don't have to download everything.

If you need more robust approach and more control, regular Algolia may be a better solution, but the setup is somewhat more complicated as you need to upload your data when you make changes to your site.

Vojtech Ruzicka's Programming Blog

Java 13 Text Blocks

String literals in Java

Other JVM languages

Raw String literals in Java

Java Raw String Literals

Multiline String literals in Java

Preview Feature

Text Blocks

Processing

Line end normalization

Indentation removal

Escaping

New String Methods

Conclusion

Detecting vulnerable dependencies in IntelliJ IDEA

Dependencies with security vulnerabilities

Detecting vulnerabilities automatically

Snyk – Detecting dependencies with known vulnerabilities

Detection directly in your IDE

Snyk plugin

Maven only

Conclusion

IDEA EduTools: Learning directly in your IDE

Getting started

Installation

Starting a course

Integrations

Stepik

Hyperskill

Coursera

Checkio

Creating courses

New course

Creating tasks

Gradle

Sharing your course

Conclusion

Analyzing dependencies in IntelliJ IDEA

Dependency Matrix

Visual aids

We need to go deeper

Cyclic dependencies

Taking Action

Analyzing Maven dependencies

Maven Helper plugin

Conclusion

Try with resources in Java

Traditional try-catch-finally

Try with resources

Decompiled example

Multiple resources

Java 9 improvements

Implementing your own resources

IntelliJ IDEA integration

Conclusion

Java By Comparison Book Review

Authors

Concept

Audience

Contents

Conclusion

Free Samples

Useful Links

Java 13 Enhanced Switch

Traditional switch

Fall-through

Single value cases

Enhanced Switch

Preview Feature

Multiple values in a case

Switch expression

Null Pointer Exception Safety

Switch with arrows

Scope

IntelliJ IDEA support

History

Future enhancements

Summary

Javascript async await tutorial