Brian J. Cardiff Francisco Tarulla 11 Dec 2019

Crystal 0.32.0 released!

Crystal 0.32.0 has been released!

This release comes with consistencies, happiness, improvements in std-lib and tools, and important changes in concurrency.

There are 197 commits since 0.31.1 by 44 contributors.

Let’s review some highlights in this release. But don’t miss out on the rest of the release changelog which has a lot of valuable information.

Language changes

The language took one more tiny step in the direction of consistency. The boolean negation method ! can now be called as a regular method call as expr.!. This kind of changes are great to avoid quirks in metaprogramming. Read more at #8445.

Macros

Other consistencies in the macro realm are the possibility to list class variables using TypeNode#class_vars, and been able to use map_with_index on ArrayLiteral and TupleLiteral. Macro lovers can find more about these changes at #8405, #8049, and #8379.

A powerful feature is that you are now able to list all types a module is directly included in by using TypeNode#includers. Read more at #8133.

Compiler

Language semantics

There was a method lookup bug fixed at #8258. You need to worry only if you have multiple overloads of the same method with a very specific combination of aliases and union types (one of them uses an alias to a union involving a type that also has an overload).

Given

alias X = Int8 | Int32

def foo(x : Int32)
  42
end

def foo(x : X)
  'a'
end

Since Crystal 0.32.0 foo(1) returns 42, instead of a.

Doc generator

The doc generator can produce a sitemap.xml which lists all HTML pages accessible for search engines. The goal is to use this sitemap to assign lower priorities to outdated doc pages. This mechanism is even better than setting a canonical url for indexed documentation. The compiler will make use of this in the near future and it might be useful for hosted documentations out there. Read more at #8348 and crystal-website#79.

As the language evolves, some conventions and features can be better advertised. For yielding methods, a non-capture block argument & will be shown in the documentation signature. Read more at #8394, and if you want to recall what the non-capture block argument is, check again #8117 from 0.31.0.

Distributions

As a heads up, the base docker image since 0.32.0 is updated to bionic and llvm-8.0. Read more at #8442.

Standard library

Attention to details contributes to happiness. There will no longer be Nil assertion failed without context for getter! and property!. The type and method information will be included for clarity.

class Foo
  getter! bar

  def initialize(@bar : Int32? = nil)
  end
end

Foo.new.bar # raises NilAssertionError: Foo#bar can't be nil

Read more at #8200 and #8296.

Spec

Be prepared for spec happiness. You can now specify code to run before, after and around the it blocks of a spec or the hole suite. You can also scope these hooks to run on a specific context or describe block. Note that variables declared inside hooks are not accessible in the it block itself, so they are aimed to play with shared context or setup resources.

The methods you will be looking for are before_each, after_each, before_all, after_all, around_each, around_all and can be used as follows:

require "spec"

describe "Users" do
  before_all do
    # setup a database
  end

  before_each do
    # truncate all tables
  end

  it "can create entity" do
    # test something assuming empty db
  end

  describe "initialized system" do
    before_each do
      # initialize some data
    end

    after_each do
      # clean up some resources
    end

    it "existing entity can work" do
      # test something assuming initialized data
    end
  end
end

Read more about spec hooks at #8302.

The happiness does not stop there. You are able to tag it block in specs with single or multiple strings that will allow you to select which ones to run using crystal spec CLI.

In a it block add a named argument tags which may contain either a String or an Array(String).

describe Foo do
  it "(1) an untagged test" do
  end

  it "(2) a fast test", tags: "fast" do
  end

  it "(3) a slow test", tags: "slow"do
  end

  it "(4) a test with a star", tags: "starred" do
  end

  it "(5) a slow test with a star", tags: %w(slow starred) do  # same as tags: ["slow", "starred"]
  end
end

Filter the specs by inclusion or exclusion.

$ crystal spec --tag fast # runs (2)
$ crystal spec --tag ~slow # runs (1) (2) (4)

Or even combine them

$ crystal spec --tag starred --tag fast # runs (2) (4) (5)
$ crystal spec --tag starred --tag ~slow # runs (4)

Please do not use tags prefixed with ~. Read more at #8068.

And, last but not least, when using should or should_not with be_a(T) or be_nil you are now able to use the result of the expression as a narrowed type and call methods that would otherwise complain due to the original union.

So, for nillable types you can do the following to avoid not_nil! along the way:

x = "42".to_i32?        # x : Int32 | Nil
x = x.should_not be_nil # update x to a narrowed type
typeof(x)               # Int32

And with any arbitrary unions, something like the following to avoid casts:

x = 1 || 'a'
typeof(x)                # => Int32 | Char
x = x.should be_a(Int32) # update x to a narrowed type
typeof(x)                # => Int32
x.to_f                   # => 1.0

Concurrency and Parallelism

There has been important work regarding concurrency and parallelism. Channel and how select is implemented got internal refactors and fixes. These changes fix the behavior on closed or closing channels which are more likely to happen with multi-thread. And there have been performance improvements along the way.

Read more about Channel internals refactor and optimizations at #8322 and #8497.

Read more about the fixes related closed Channel at #8284, #8249, #8304.

Mutex also got some improvements, both feature- and performance-wise. Read more about them in #8295 and #8563. The Mutex as you may know prevents multiple fibers running their critical sections concurrently. This is independent of whether the fibers run in the same or in different threads. There are three behaviors or protection levels the mutex supports. When creating a Mutex you might specify which protection level to use: Mutex.new(:checked) (default), Mutex.new(:reentrant) or Mutex.new(:unchecked).

A :checked mutex provides deadlock protection. Attempting to re-lock the mutex from the same fiber will raise an exception.

The :reentrant protection maintains a lock count allowing it to be used in recursive scenarios. Attempting to unlock an unlocked mutex, or a mutex locked by another fiber will raise an exception.

You can disable all protections with :unchecked. This is particularly useful for some scenarios where the lock and unlock of a critical section need to occur in different fibers.

Text

String interpolations are widely used in the language. The std-lib is updated with a String.interpolation method that will be used directly by the compiler. Up to 0.31.1 "hello #{world}!" was a syntax-sugar of

String.build do |io|
  io << "hello "
  io << world
  io << "!"
end

But is now changed to

String.interpolation("hello ", world, "!")

This subtle change allows performant specialized interpolation logic allowing to forget about "foo#{bar}" vs "foo" + bar.

There is a small breaking change though "#{str}" returns the same string instance stored in str. But since String is immutable you should not worry about that change. Read more at #8400.

For input parsing we have these cool new methods: String#presence (and Nil#presence). Here is an example of what they will let us do:

puts "a".presence || "default" # => "a"
puts nil.presence || "default" # => "default"

Read more at #8345, #8508

We will stop supporting String#codepoint_at in favor of String#char_at(index).ord. Read more at #8475.

Collections

We won’t be using Enumerable#grep anymore. Now we are just using Enumerable#select.

So, instead of:

puts ["Foo", "Bar", "Baz"].grep(/^B/) # => ["Bar", "Baz"]

We are going to use:

puts ["Foo", "Bar", "Baz"].select(/^B/) # => ["Bar", "Baz"]
# or
puts ["Foo", "Bar", "Baz"].select {|word| word.starts_with?("B")} # => ["Bar", "Baz"]

Read more at #8452.

With this version we may tell a Hash (or a Set) to compare keys by object_id. After calling compare_by_identity how the receiver hash behaves will change. Read more at #8451.

h1 = {"foo" => 1, "bar" => 2}
h1["fo" + "o"]? # => 1

h1.compare_by_identity
h1.compare_by_identity? # => true
h1["fo" + "o"]?         # => nil # not the same String instance

Iterating over an array with chunks of 2 elements? Well, we have a treat for you. Now we may use Enumerable#each_cons_pair and Iterator#cons_pair with rocket-enhanced performance! Read more at #8332.

Serialization

While using JSON.mapping, YAML.mapping, JSON::Serializable and YAML::Serializable sometimes the data is not quite in the right format or doesn’t quite match the type we expect. The converter option allows you to inject some logic while converting from/to the different format. There are some new awesome helper modules: JSON::ArrayConverter, YAML::ArrayConverter, JSON::HashValueConverter that will allow to define the converters on the items or values to be used. Read more at #8156.

If the above does not make you happy enough, wait until you discover use_json_discriminator and use_yaml_discriminator, that will allow you to specify which concrete type to use, based on a property value. Read more at #8406.

Breaking news! XML::Reader#expand will raise an error, and if we want the old behavior then we have XML::Reader#expand? Making things more consistent! Read more at #8186.

Files

Breaking news! File.expand_path and Path#expand will no longer expand home (~) by default. It is now an opt-in argument: home: true or even `home: “/use/this/as/home”. Read more at #7903.

DB

crystal-lang/crystal-db got a new release: 0.8.0. The shiny new feature is the DB::Serializable module and DB::Field annotation matching the JSON and YAML counterparts. Read more at crystal-db#115.

If you want to use that feature be sure to upgrade to a driver that require 0.8.0 at least.

Next steps

Please update your Crystal and report any issues. We will keep moving forward and start the development focusing on 0.33.

It will also be helpful if your shards are run against Crystal nightly releases. Either Docker or Snap are the current channels to get them easily. This will help reduce the friction of a release while checking if the ecosystem is in good shape.

We have been able to do all of this thanks to the continued support of 84codes, and every other sponsor. It is extremely important for us to sustain the support through donations, so that we can maintain this development pace. OpenCollective and Bountysource are two available channels for that. Reach out to crystal@manas.tech if you’d like to become a direct sponsor or find other ways to support Crystal. We thank you in advance!

Crystal 0.32.0 has been released!

This release comes with consistencies, happiness, improvements in std-lib and tools, and important changes in concurrency.

There are 197 commits since 0.31.1 by 44 contributors.

Let’s review some highlights in this release. But don’t miss out on the rest of the release changelog which has a lot of valuable information.

Language changes

The language took one more tiny step in the direction of consistency. The boolean negation method ! can now be called as a regular method call as expr.!. This kind of changes are great to avoid quirks in metaprogramming. Read more at #8445.

Macros

Other consistencies in the macro realm are the possibility to list class variables using TypeNode#class_vars, and been able to use map_with_index on ArrayLiteral and TupleLiteral. Macro lovers can find more about these changes at #8405, #8049, and #8379.

A powerful feature is that you are now able to list all types a module is directly included in by using TypeNode#includers. Read more at #8133.

Compiler

Language semantics

There was a method lookup bug fixed at #8258. You need to worry only if you have multiple overloads of the same method with a very specific combination of aliases and union types (one of them uses an alias to a union involving a type that also has an overload).

Given

alias X = Int8 | Int32

def foo(x : Int32)
  42
end

def foo(x : X)
  'a'
end

Since Crystal 0.32.0 foo(1) returns 42, instead of a.

Doc generator

The doc generator can produce a sitemap.xml which lists all HTML pages accessible for search engines. The goal is to use this sitemap to assign lower priorities to outdated doc pages. This mechanism is even better than setting a canonical url for indexed documentation. The compiler will make use of this in the near future and it might be useful for hosted documentations out there. Read more at #8348 and crystal-website#79.

As the language evolves, some conventions and features can be better advertised. For yielding methods, a non-capture block argument & will be shown in the documentation signature. Read more at #8394, and if you want to recall what the non-capture block argument is, check again #8117 from 0.31.0.

Distributions

As a heads up, the base docker image since 0.32.0 is updated to bionic and llvm-8.0. Read more at #8442.

Standard library

Attention to details contributes to happiness. There will no longer be Nil assertion failed without context for getter! and property!. The type and method information will be included for clarity.

class Foo
  getter! bar

  def initialize(@bar : Int32? = nil)
  end
end

Foo.new.bar # raises NilAssertionError: Foo#bar can't be nil

Read more at #8200 and #8296.

Spec

Be prepared for spec happiness. You can now specify code to run before, after and around the it blocks of a spec or the hole suite. You can also scope these hooks to run on a specific context or describe block. Note that variables declared inside hooks are not accessible in the it block itself, so they are aimed to play with shared context or setup resources.

The methods you will be looking for are before_each, after_each, before_all, after_all, around_each, around_all and can be used as follows:

require "spec"

describe "Users" do
  before_all do
    # setup a database
  end

  before_each do
    # truncate all tables
  end

  it "can create entity" do
    # test something assuming empty db
  end

  describe "initialized system" do
    before_each do
      # initialize some data
    end

    after_each do
      # clean up some resources
    end

    it "existing entity can work" do
      # test something assuming initialized data
    end
  end
end

Read more about spec hooks at #8302.

The happiness does not stop there. You are able to tag it block in specs with single or multiple strings that will allow you to select which ones to run using crystal spec CLI.

In a it block add a named argument tags which may contain either a String or an Array(String).

describe Foo do
  it "(1) an untagged test" do
  end

  it "(2) a fast test", tags: "fast" do
  end

  it "(3) a slow test", tags: "slow"do
  end

  it "(4) a test with a star", tags: "starred" do
  end

  it "(5) a slow test with a star", tags: %w(slow starred) do  # same as tags: ["slow", "starred"]
  end
end

Filter the specs by inclusion or exclusion.

$ crystal spec --tag fast # runs (2)
$ crystal spec --tag ~slow # runs (1) (2) (4)

Or even combine them

$ crystal spec --tag starred --tag fast # runs (2) (4) (5)
$ crystal spec --tag starred --tag ~slow # runs (4)

Please do not use tags prefixed with ~. Read more at #8068.

And, last but not least, when using should or should_not with be_a(T) or be_nil you are now able to use the result of the expression as a narrowed type and call methods that would otherwise complain due to the original union.

So, for nillable types you can do the following to avoid not_nil! along the way:

x = "42".to_i32?        # x : Int32 | Nil
x = x.should_not be_nil # update x to a narrowed type
typeof(x)               # Int32

And with any arbitrary unions, something like the following to avoid casts:

x = 1 || 'a'
typeof(x)                # => Int32 | Char
x = x.should be_a(Int32) # update x to a narrowed type
typeof(x)                # => Int32
x.to_f                   # => 1.0

Concurrency and Parallelism

There has been important work regarding concurrency and parallelism. Channel and how select is implemented got internal refactors and fixes. These changes fix the behavior on closed or closing channels which are more likely to happen with multi-thread. And there have been performance improvements along the way.

Read more about Channel internals refactor and optimizations at #8322 and #8497.

Read more about the fixes related closed Channel at #8284, #8249, #8304.

Mutex also got some improvements, both feature- and performance-wise. Read more about them in #8295 and #8563. The Mutex as you may know prevents multiple fibers running their critical sections concurrently. This is independent of whether the fibers run in the same or in different threads. There are three behaviors or protection levels the mutex supports. When creating a Mutex you might specify which protection level to use: Mutex.new(:checked) (default), Mutex.new(:reentrant) or Mutex.new(:unchecked).

A :checked mutex provides deadlock protection. Attempting to re-lock the mutex from the same fiber will raise an exception.

The :reentrant protection maintains a lock count allowing it to be used in recursive scenarios. Attempting to unlock an unlocked mutex, or a mutex locked by another fiber will raise an exception.

You can disable all protections with :unchecked. This is particularly useful for some scenarios where the lock and unlock of a critical section need to occur in different fibers.

Text

String interpolations are widely used in the language. The std-lib is updated with a String.interpolation method that will be used directly by the compiler. Up to 0.31.1 "hello #{world}!" was a syntax-sugar of

String.build do |io|
  io << "hello "
  io << world
  io << "!"
end

But is now changed to

String.interpolation("hello ", world, "!")

This subtle change allows performant specialized interpolation logic allowing to forget about "foo#{bar}" vs "foo" + bar.

There is a small breaking change though "#{str}" returns the same string instance stored in str. But since String is immutable you should not worry about that change. Read more at #8400.

For input parsing we have these cool new methods: String#presence (and Nil#presence). Here is an example of what they will let us do:

puts "a".presence || "default" # => "a"
puts nil.presence || "default" # => "default"

Read more at #8345, #8508

We will stop supporting String#codepoint_at in favor of String#char_at(index).ord. Read more at #8475.

Collections

We won’t be using Enumerable#grep anymore. Now we are just using Enumerable#select.

So, instead of:

puts ["Foo", "Bar", "Baz"].grep(/^B/) # => ["Bar", "Baz"]

We are going to use:

puts ["Foo", "Bar", "Baz"].select(/^B/) # => ["Bar", "Baz"]
# or
puts ["Foo", "Bar", "Baz"].select {|word| word.starts_with?("B")} # => ["Bar", "Baz"]

Read more at #8452.

With this version we may tell a Hash (or a Set) to compare keys by object_id. After calling compare_by_identity how the receiver hash behaves will change. Read more at #8451.

h1 = {"foo" => 1, "bar" => 2}
h1["fo" + "o"]? # => 1

h1.compare_by_identity
h1.compare_by_identity? # => true
h1["fo" + "o"]?         # => nil # not the same String instance

Iterating over an array with chunks of 2 elements? Well, we have a treat for you. Now we may use Enumerable#each_cons_pair and Iterator#cons_pair with rocket-enhanced performance! Read more at #8332.

Serialization

While using JSON.mapping, YAML.mapping, JSON::Serializable and YAML::Serializable sometimes the data is not quite in the right format or doesn’t quite match the type we expect. The converter option allows you to inject some logic while converting from/to the different format. There are some new awesome helper modules: JSON::ArrayConverter, YAML::ArrayConverter, JSON::HashValueConverter that will allow to define the converters on the items or values to be used. Read more at #8156.

If the above does not make you happy enough, wait until you discover use_json_discriminator and use_yaml_discriminator, that will allow you to specify which concrete type to use, based on a property value. Read more at #8406.

Breaking news! XML::Reader#expand will raise an error, and if we want the old behavior then we have XML::Reader#expand? Making things more consistent! Read more at #8186.

Files

Breaking news! File.expand_path and Path#expand will no longer expand home (~) by default. It is now an opt-in argument: home: true or even `home: “/use/this/as/home”. Read more at #7903.

DB

crystal-lang/crystal-db got a new release: 0.8.0. The shiny new feature is the DB::Serializable module and DB::Field annotation matching the JSON and YAML counterparts. Read more at crystal-db#115.

If you want to use that feature be sure to upgrade to a driver that require 0.8.0 at least.

Next steps

Please update your Crystal and report any issues. We will keep moving forward and start the development focusing on 0.33.

It will also be helpful if your shards are run against Crystal nightly releases. Either Docker or Snap are the current channels to get them easily. This will help reduce the friction of a release while checking if the ecosystem is in good shape.

We have been able to do all of this thanks to the continued support of 84codes, and every other sponsor. It is extremely important for us to sustain the support through donations, so that we can maintain this development pace. OpenCollective and Bountysource are two available channels for that. Reach out to crystal@manas.tech if you’d like to become a direct sponsor or find other ways to support Crystal. We thank you in advance!

comments powered by Disqus