hussainweb.me

Porting token module to Drupal 8

Discovery

I have contributed to Drupal 8 over the course of last couple of years. I have also written custom modules for Drupal 8, but I never really tried building a Drupal 8 site. After Drupal 8’s release last November, I thought it was time to change that.

That is how I set about upgrading one of my Drupal 7 sites to Drupal 8. As soon as I started building a list of modules, I stopped. Pathauto wasn’t ready yet. It was not even close to being ready. Sure, there is a 8.x branch but also a warning that the architecture may change significantly. Okay, it was time to roll up my sleeves and get to work. I cloned the github repo where the port was happening and started.

And then I stopped again. It needed token, and it was also just a dev release. Now, token is a module everyone needs and no one knows, especially after token is mostly in core since Drupal 7. How can that still be dev two months after Drupal 8 final?! Okay, let’s start with token. And that is the story of how I landed in token’s issue queues one fine Monday.

Requirements

I started with reviewing the token module code. It seemed like it worked with Drupal 8 but it was still essentially Drupal 7 based architecture. I am not saying there is anything wrong with Drupal 7 based architecture but we were missing out on the clean code, discrete responsibilities, and easy testing that comes with code architected for modern PHP.

Refactoring

I found an issue that covered moving helper functions to services but the response wasn’t great. I see the point. The code worked and refactoring was not a priority against getting it to work; however, to me, getting it to work cleanly along with rest of Drupal would need a refactored code base. It was not easy getting in either, what with over 3000 combined lines of code in module and inc files. I started looking for low hanging fruits and found one – token browser not shown on help page.

Theme Hooks

After that and other simple fixes, I looked into converting theme hooks to TWIG. Theme hooks work in Drupal 8 but are deprecated. Further, there were three theme hooks in token module and they were interdependent and confusing. I started with eliminating one of them and also submitted patches or pull requests to other modules that used this behaviour.

Token tree screenshot

Essentially, there was a theme hook called token_tree_link which was never used. Instead, if a module wanted to show a link to show a token tree, they should use the token_tree hook with ‘#dialog’ option set to TRUE. This was quite counter-intuitive in my opinion and also it did not allow you to show links without dialog (we subsequently removed the dialog option entirely as there was no use case to show it without a dialog). In other words, the token_tree hook was doing too much and I submitted a patch that removed this functionality. Now, you should use token_tree_link directly if you want to show a link. Since this would break many other projects (like pathauto, metatag, etc…), I submitted patches to many of them.

Eventually, the token_tree_link theme hook was converted to a template_preprocess along with a twig file, as per current Drupal 8 standards.

As it turned out, the theme hook token_tree did little in itself and we later refactored it as a method on tree builder service. There was another hook called tree_table and that was converted to a render element. Now, to show a token tree, all you need to do is:


$element = [
 '#type' => 'token_tree_table',
 '#token_tree' => $token_tree, // Generated token tree.
 ...
];

The token_tree theme hook, which was a convenient function to generate the token tree (and hence not really just a theme hook) was moved to the buildRenderable() method on the token.tree_builder service.

Services

The first helper function to move to a service was token_get_info(). We didn’t just move it, but also simplified the function greatly. The function token_get_info() would return different results depending on different parameters and that is not a good idea. We split the function into three different methods – getInfo(), getTypeInfo() and getTokenInfo(). Similarly, we deprecated other functions and moved them to methods. Here is a list of functions now in Token service.

  • token_get_info — getInfo()
  • token_get_info — getTypeInfo()
  • token_get_info — getTokenInfo()
  • token_get_global_token_types — getGlobalTokenTypes()
  • token_get_invalid_tokens_by_context — getInvalidTokensByContext()
  • token_get_invalid_tokens — getInvalidTokens()

The cool thing here was that we replaced the core’s Token service with the one provided by the token module and it made a lot of sense. This also means that if you have the token module enabled, all you need to do is call \Drupal::token() to get the token service.

The next set of functions to be removed were related to building token trees. These functions were converted to corresponding methods on token.tree_builder service.

  • token_build_tree — buildTree()
  • token_flatten_tree — flattenTree()
  • _token_build_tree — getTokenData() [internal use only]

Later, we also moved the token_tree theme hook functionality to a method on this service – buildRenderable(). This was done because the theme hook did a lot of processing to generate the token tree and pass it on to tree_table theme hook, which was really out of the scope of a theme hook.

Next, we moved token entity mapping functions to it’s own service.

  • token_get_entity_mapping — getEntityTypeMappings()
  • token_get_entity_mapping('token') — getEntityTypeForTokenType()
  • token_get_entity_mapping('entity') — getTokenTypeForEntityType()

This was another function that would give different kinds of results depending on the first parameter. For cleanliness, we just split them into different methods.

The above functions were deprecated for a while and subsequently removed.

Devel Integration

Token had stopped working with Devel as well during its development. We used devel’s ideas of a local task derivative and route subscriber to add dynamic routes to show tokens for each of the entities. There was already a controller responsible for showing the page with all tokens which was refactored to work with all entity types.

Tests

Apart from writing tests for the new services that we introduced, we also fixed a lot of old tests. Some of the tests were integration tests which were using simpletests’ KernelTestBase. We changed all such tests to use the modern Drupal\KernelTests\KernelTestBase which brought in some speed improvements as well. These can be run directly using PHPUnit.

Other Changes

There were a lot of other small changes that went a long way in improving the module, such as replacing define with class constants, removing deprecated functions, cleaning up a lot of old code, At the time of this writing, we are at alpha2 and there is still some work left, particularly around render caching token trees. Please help getting token to stable.

I hope this helps you in refactoring your module to Drupal 8, which are just modern PHP principles. Please let me know your feedback, questions, or suggestions.

hw

, , ,