Fix Your Flaky Faker Tests

Github CI failed again - You check the failing test and are greeted with a familiar error message:

Assertion with =~ failed
code:  assert html =~ user.name
left: # giant blob of html
right: "D'angelo"

Upon investigating, you see that the name is in fact present in the HTML, but it is HTML encoded as D'angelo

Floki, the HTML parser used by LiveViewTest, HTML encodes certain characters by default.

Fortunately, the fix is just one line:

# config/test.exs
config :floki, :encode_raw_html, false

Now, your tests won’t HTML encode characters such as ><&'" - and your assertions will pass.

Why does this happen?

Faker is a popular library for generating real-looking fake data. Using functions like Faker.Person.first_name() can be a good way to seed test data into the application. Sometimes, Faker will generate the names of things with certain special characters. The most common I see is first or last names that contain apostrophes.

Floki is the HTML parser used by LiveViewTest under the hood. When invoking functions like render/1, Floki parses the HTML and - importantly - HTML encodes some special characters.

When asserting against html that should contain the name of something, these two properties can collide and cause test failures. Technically, this is false:

assert "O'Shaugnessy" == "O'Shaugnessy"

That, of course, isn’t what we really meant to say with our test assertion.

The Fix

The fix is this single line in config/test.exs:

# config/test.exs
config :floki, :encode_raw_html, false

This option is a little bit buried under the documentation for raw_html/2 in the Floki docs. It states that encoding special HTML entities can be ignored by passing it as an option.

To control the behavior at an individual function level, the :encode option can be passed to raw_html/2, if passing it to every invocation is undesirable:

Floki.raw_html(html_tree, encode: false)

Controller Tests

This will not work for “dead”-view tests that use html_response/2. ConnTest does not use Floki and the config option will not work.

Instead, consider a small function in your test helper suite to html encode values you are asserting are contained in the HTML:

def html_encode(html) when is_binary(html) do
  html
  |> Phoenix.HTML.html_escape()
  |> Phoenix.HTML.safe_to_string()
end