Block-Based Themes and the Problem with Dynamic Data in HTML Templates
The Gutenberg project and its eventual full-site editing feature is coming upon a major issue that will need to be solved. Block-based themes of the future are currently on a path toward a template system that will comprise of plain HTML files. While that will work for the majority of a theme’s output, the trouble is figuring out how the project will handle dynamic values.
Most of the discussion has centered on handling URLs, which are probably the most common use case. Currently, theme templates have all sorts of dynamic content. Much of that will be replaced with blocks as we continue moving toward full-site editing. However, not all dynamic data will have a block equivalent.
A good example is that theme authors cannot currently add the homepage URL to the navigation block. Some experimental block-based themes are using a simple / character, which points to the wrong location on many WordPress installs.
Solving this issue sooner rather than later is important for the progression of theme development in a block world. However, such a solution needs to be carefully crafted so that the theming community is not bogged down by a decade or more with a poor templating implementation.
The Current Proposals
The Gutenberg repository currently has an open ticket for discussion on handling dynamic values in templates. At the moment, there are four proposals on how to address the issue.
On-the-Fly String Replacement
One solution would be to use PHP to parse each HTML file and replace strings representing dynamic data on the fly. This would require parsing all of a theme’s templates on every page load. The downside is that it would slow down the page load. We would need real unit tests to see how much of a spike in loading time this method creates.
Assuming a Mustache-like syntax, templates would have values such as the following image output:
<img src="{{ theme_image_example }}" />One added benefit of adopting such a solution is that WordPress could automatically escape these dynamic values by default. This would be a boon to theme security, which is one of the biggest issues faced by the theme review team.
One-Time String Replacement
The second solution proposes using the same method but parsing the HTML files once, upon theme activation, and replacing dynamic values with proper values. The largest benefit of this method is that the parsing would not affect front-end loading speed.
This method is problematic because it does not account for changes to templates after the initial parsing. It also does not handle scenarios when a value changes via user input. For example, a user may decide to change the location of their blog posts page. Therefore, a parsed URL that becomes static would point to the wrong location.
Templates as JSON
A third solution proposes the idea of turning theme files into JSON. It is far easier to parse and extract data from a JSON file than and HTML file. However, theme designers do not write JSON to build template output. HTML exists for a reason.
This solution would raise the barrier so high for new theme authors that it would be rare for those who just learned basic CSS and HTML to get into WordPress theme development. This idea is so foreign to the idea of template design, that it should not be a serious consideration.
Templates That Return Blocks via PHP
The fourth and final proposal is to use PHP files with a function that returns a block template. This method would be straightforward and could be picked up easily for existing theme authors who have a familiarity with PHP.
A template would look something like the following:
function my_theme_front_page() {
return '<!-- wp:cover {"url":"' . get_template_directory_uri() .'/block-background-image.png","id":273,"dimRatio":0,"minHeight":647,"align":"wide"} -->';
}This idea puts more focus on PHP than HTML. It would be the easiest-to-implement solution for the Gutenberg development team. However, like the JSON method, it would raise the barrier to entry for first-time theme authors. It will mean making sure quote and double-quote characters do not get mixed up. The method would be prone to bugs and looks alien to modern-day templating.
Templating Should Focus on HTML
Templating should always be HTML-first. Even in our current theme system, theme authors can build beautiful, safe, and functional themes by simply knowing HTML and CSS. PHP is secondary, especially when it comes to the templating aspect. Our templating system relies on knowing HTML and plugging in a few template tags, which are PHP functions that WordPress provides for dropping in between HTML tags. This simplicity is, in part, what made WordPress theme development so easy to learn for anyone willing to put in a little time.
Block-based themes have the potential to drop the barrier even lower than our current templating system. However, templates as JSON or PHP functions runs counter to that. Any solution that pushes us farther away from the basic building blocks of the web, HTML, should not be on the table for discussion.
It may be time to adopt a proper PHP templating engine. There are plenty of examples out there. Twig, Blade, Smarty, and others have existed for years. Those also have some barrier to entry in the form of new syntax, but this should be no tougher than learning to plug template tags into the current system.
At the very least, we will need to figure out a solution for handling dynamic data in what is essentially static HTML files.
34 Comments
Interesting post Justin.
It doesn’t seem too far-fetched to me that that at some point theme author’s are going to have to start dipping their toes, feet, legs deeper and deeper into learning how to build blocks for the new editor. Which means learning a lot of javascript and subsequently JSON. Being primarily a frontend WordPress dev specializing in javascript and react, I am very biased, but I have to believe if we’re moving toward block-based themes, why wouldn’t we go all in by using templates as JSON?
Report
JSON based theme templates could take WordPress themes forward to a much better future. Excluding this option from serious consideration ( as suggested) is just not appropriate & I’m confident the Gutenberg team won’t do that.
Report
The barrier to entry is why I do not think we will or should move that direction. Burdening first-time theme authors with learning how JSON works as opposed to basic HTML, which they will likely already be familiar with, would mean that only more advanced developers would be building themes. WordPress has always had a low barrier to getting into theme development. It has long been a part of its recipe for success. I’m not sure if moving away from that would be beneficial for the project.
Now, if the suggestion is that we move 100% into visual theme building and away from hand-coding them, that may change.
Report
There are those who say PHP is a proper templating engine. To which I generally agree.
If more WordPress developers would embrace HEREDOC syntax in PHP, this would be far less of an issue. #jmtcw
BTW, our team is currently building a solution in Gutenberg that leverages PHP for generating dynamic content for which we need to be feature complete by end of March. Wish us luck…
Report
Good luck on the project! And, good points too.
I actually started this entire post off as a discussion about this but decided to go in a different direction as I continued diving into the piece. I tend to agree and have been one of those devs who has pushed back a bit against adopting any sort of templating engine. With that said, I have come around on the idea a bit more in the past couple of years on some projects. I do think it’s worth evaluating while we’re changing the template system around.
Report
Thanks Justin.
Yeah, I’m certainly not religious about PHP vs. templating engines. But then again…
So we are ripping out the use of Mustache for PHP and replacing with pure PHP. Why?
Debugging Mustache is like flying in the cargo hold vs. debugging PHP with PhpStorm or VSCode is like flying first class.
When using Mustache — which is admittedly worse that using templates that support logic — you have to populate an array of arrays and
stdClassobjects for damn near every permutation you can imagine, and it ends up being a maintenance nightmare.Thirds, and admittedly by far the least important in most cases is performance. Using templating can be slower than just doing it in PHP, unless the templating system has a compile to PHP option and you can implement a build process into your workflow.Yes, I know there are techniques to improve #2 but given #1 we felt it was better to just go back to PHP.
I am just posting this as a cautionary tale for anyone who considers using a templating language. Better to know this and still choose to use a templating language than to not know this, commit to using a templating language on a larger project, and later become surprised with these issues after it is too late to reverse course.
#fwiw. #ymmv
-Mike
P.S. If PhpStorm were to implement a single-step debugging and variable inspection feature for one or more templating engines, I might change my tune completely.
Report
Your KSES stripping out my ordered list! Here it is again:
Debugging Mustache is like flying in the cargo hold vs. debugging PHP with PhpStorm or VSCode is like flying first class.
2. When using Mustache — which is admittedly worse that using templates that support logic — you have to populate an array of arrays and
stdClassobjects for damn near every permutation you can imagine, and it ends up being a maintenance nightmare.3. Third, and admittedly by far the least important in most cases is performance. Using templating can be slower than just doing it in PHP, unless the templating system has a compile to PHP option and you can implement a build process into your workflow.
Report
For the love of God, drop underscores and move to Roots. The only thing that has kept me from using Roots is widespread adoption as a ‘standard’. Blade based components align with the block model, or am I missing something.
Report
You aren’t missing anything mate. This gutenbergization of wordpress is nothing but turning it into something similar to laravel cms with “wordpress backend”. it is not going to be laravel neither wordpress.
justin is right about the issues facing the gutenberg blocks but i think he is forgetting that the popularity of wordpress lies behind mass adoption by people who don’t know how to code at all. wordpress was, install it in few clicks engine. but now, its getting more and more complex as even the template companies and individual designers have to learn javascript to make a simple text appear on front-end in this block crazy wordpress world.
anyhow, back to the topic. I think they should take some bits from amp-bind & amp-form relationship that makes it easier to use dynamic data in AMP. it might be the easiest solution to the problem written by the author. amp-bind might be the answer.
Report
That was my point in the article. WordPress’ low barrier to entry helped it gain popularity. Even on the theme development side of things, that barrier only required some rudimentary HTML, CSS, and copy/paste skills. That meant a whole lot of the users who did not know how to code eventually found a path toward becoming a developer.
In the past decade, it’s become tougher to get into something that was once as simple as building a basic theme. But, the upcoming block-based theme system can be our chance to recapture some of that simplicity if we do not venture too far away from the basics.
Report
That’s exactly how I got into it 11 years ago. I didnt even know html.
Report
“In the past decade, it’s become tougher to get into something that was once as simple as building a basic theme. But, the upcoming block-based theme system can be our chance to recapture some of that simplicity if we do not venture too far away from the basics.”
This.
Report
Twig is not too difficult to understand and implement and it seems flexible. You just reviewed the Meta Box Views extension that uses Twig to create theme templates. Adopting Twig makes sense to me.
Report
Is there a basic explanation of the premise here? I dont understand the issue of dynamic content in themes… we build blocks and use php to get dynamic content. The user puts those blocks wherever s/he likes… how does that change with theme dev?
Report
Not all theme output is controlled by the user. And, if/when we do get to a point where the user can change anything on the front end, theme authors still need to be able to set defaults.
One example provided in the post is the navigation menu. Let’s say a theme adds a nav menu block to the header. It may want to populate that with a few links. One of those links might be a “Home” link. Currently, that requires the theme use the
home_url()PHP function. However, because templates are HTML, we are not running PHP. Therefore, the function would not work. We need a method of correctly outputting that URL.Another example might be adding a cover block to a custom front page template. The theme author will likely want to include a background image for that cover block. Right now, with HTML templates, the only way to use an image for that block is to do so with a static URL. Instead, theme authors need to be able to point to the correct image URL within the theme.
I hope that helps explain a bit more. There’s a whole host of similar scenarios that require dynamic values.
Report
Thank you! I didnt realize going to gutenberg themes meant html strictly… I suppose I’m used to combining html/php in php files.
Report
I meant to link to an earlier post, Initial Documentation for Block-Based Themes Proposed, that explains how they will work. Of course, so much of this could change in the coming months. I just went back and added the link into the post though for reference.
Some other posts that might be worth digging into that touch on the subject:
Report
As someone who uses Timber Library that adds TWIG parser to WordPress, I hope they can adopt it.
They seem to be concerned about confusing first-time theme author, but the posts loop is one of the most confusing thing in WordPress when I started learning. TWIG can solve that.
Report
Adding dynamic data to Gutenberg will be one of the best thing ever happened to the plugin. It will allow developers to connect with different data sources and expand the use of Gutenberg for other purposes.
I probably vote for the on-the-fly rendering. With a proper cache layer and lazy-load method, we might make it work without affecting the performance.
Using a template engine might work, but I don’t think WP will do that. It’s too much for this. And it might affect the whole theme ecosystem and will introduce another way to build themes (which is confusing).
Report
It’s strange to see this issue dealt with more than a year after Gutenberg was merged, and a lot more than two years into the project. I mean, dynamic data is an essential part o WordPress, IMO.
Report
Exactly this. Should have been solved at initial concept stage years ago. Now lots of breaking changes will have to be added, again and affect millions of live websites…
Report
It wasn’t a key issue that needed to be solved while the block system was simply a content editor. However, as it moves into being a full-site editor that is merged with the theme template system, it becomes a hurdle that must be cleared.
Report
Gutenberg blocks are JSX-based. In the interest of consistency, it seems like templates should be written the same way.
Report
One would think that this could easily be solved by shortcodes, which already exist and are in wide use.
Report
I think there’s an important missing piece in this conversation that seems to have gone into a battle between templating engines.
One of the key features is being able to save/tweak/fork and share the templates visually (using the block editor).
Another missing piece is that the html as written today is already dynamic. Blocks are dynamic. There’s no difference between a block HTML comment and say a twig token, both can augment HTML with dynamic features, the difference is that the blocks comments can be edited visually while for the twig tokens, you need to write code.
Does this mean this issue could be solved by just having some “inline blocks” or “inline tokens” that the editor can understand somehow, maybe? maybe not? I don’t have a precise answer now, but editing capabilities is an important piece of the puzzle.
Report
Yes, it’s unfortunate that we veered too far over into discussion on templating engines. I thought it was important to bring up but didn’t want it to be the focus. Mostly, I just want to get the ideas churning.
I do like the idea of “inline blocks” or tokens or whatever. Admittedly, it would seem a little odd to use the current block syntax inline. I’m not sure what that would look like if we go that route.
Anyway, interesting food for thought.
Report
I’m confused. Do I need to install the Gutenberg plugin even though I have Gutenberg running from WordPress? In another words, I have block plugins installed like Stackable and really like but am confused as to weather or not I need the actual Gutenberg plugin itself. Thanks so much.
Report
Do not worry about being confused. The terminology around the block editor and Gutenberg often gets muddied. It may even be worth doing a post that dives deeper into this subject. But, here’s the short version:
The editor you have running on your website since WordPress 5.0 is called the block editor.
Gutenberg is the name of the project and plugin where development of the block editor (and beyond) happens before it lands in WordPress. It’s basically a codename for the block editor project. When we talk about future features, these are things that are generally already in progress within the Gutenberg plugin but not WordPress itself.
You absolutely do not need to install the Gutenberg plugin unless you want to test the latest features before they are in WordPress. I highly recommend using it because you will always have access to the latest and greatest features, sometimes months in advance. With that said, there are sometimes theme incompatibilities when running the plugin. The plugin is stable and not quite bleeding-edge, but you should keep in mind that sometimes things may break. We run the plugin here on WP Tavern because I like to use the newest stuff.
Report
I’m a theme designer/developer but I work directly with end-users developing custom themes. I feel like I’m still missing something here. I’ve read posts linked from this one and other resources and still don’t understand why .html files in the theme folder are desired for Gutenberg other than a strange notion to all of a sudden upheave the WP ecosytem for the sake of making theme development more accessible. If we’re going to do such an upheaval, we should go towards a modern app development workflow that most professional developers would benefit from (and make WP less of a cul-de-sac of a career pathway for aspiring developers). It’s funny that the working development branch is itself more in line with a modern workflow for the sake of the project devs.
Any way, why not continue with PHP files that are mostly HTML with PHP for dynamic content? If the proposal is for developers to work on the logic side of things in the WP admin, that sounds terrible–especially if there is no backwards compatibility for the current theming system.
As an aside, for my work, the block system is nearly pointless altogether (especially with ACF as an option). I never want to go through the trouble of designing a site just to empower my client to severely diminish it (which, anytime they go about rearranging the layout, is nearly 100% likely). Empowering the client to change content, not change design is our motto. Honestly, any time I setup a third party theme for a client on a budget, I always stick with the thought-out demo designs from the theme developer. Making a truly functional and beautiful design takes a lot of time and effort. Even then, theme builders end up making it much easier for the end user to accidentally mess up a design, even when they’re just trying to change content.
It’s also common for me to have clients that come to me after trying to “design” their site via some theme builder. They are usually frustrated and the results are lackluster, to put it mildly.
I get helping the SEO/Graphic Designer-type of professional be more empowered to create themes on the open source side of WP. But apart from that, the entire Gutenberg project seems like a way for Automattic to compete with Squarespace and geeky hobbyists to theorize about the democratization of the internet. Both impulses are understandable, but how long can one codebase support these end-goals and the goals of professional developers?
Report
Gutenberg devs obviously have “one-user-is-admin” sites in mind who spend time to fiddle around with the content again and again and update this and that every few weeks.
In reality most sites are not like this.
Most sites are designed by some agency and clients have a limited author account or similar to upload a new menu pdf or change some opening hours maybe once or twice a year. Those users will break everything if you allow them to use Gutenberg as admin, not because they want to redesign things and can’t but just by accident.
Most clients invest an amount X to “have a website made”. They do not have nor invest another amount X for continuous required technical changes and theme updates and whatnot so some full site editing feature works with whatever plugins which they both will never really use.
Thats the problem.
Report
Another perspective on Gutenberg.
Gutenberg is an awesome foundation for a corporate website that allows a team of developers to keep the internal end user from screwing up the design but still provide the internal end user a tremendous amount of business flexibility to achieve their revenue goals.
Specifically, we are building a Gutenberg-based framework to be used by many different similar websites as the business acquires similar businesses, keep the brands of the businesses they acquire, and then builds them a brand-specific website using our common codebase.
We are building full-width blocks that their internal end users can drag and drop to create a quick page layout. We are giving the end-users about 10% as much flexibility as most “one-user-is-admin” WordPress site owners get so there is no way they can mess up the design.
And you know what? It is working fabulously.
Simply put Gutenberg is a great foundation for developers to build bespoke business-specific blocks to empower task-oriented end-users who just want to get their jobs done. And it keeps us from having to create our own UI for page building, and it keeps us from having to extend a 3rd party page builder that tries to do far too much and is thus overly complicated.
#fwiw
P.S. All that said, this is not an inexpensive approach. Our project is probably well over six figures thus far.
Report
Thanks for that perspective. I’m glad to hear a success story. Have you tried Advanced Custom Fields? It’s amazing. It handles the UI in the admin area and integrates fairly simply into the front end. It has many types of fields it can add to the backend, but one in particular is the Flexible Content field which does something similar to what you’re using Gutenberg for. It would be interesting to compare the workflow for using that field with Gutenberg blocks for full-width content and other applications.
If for some reason you haven’t heard of ACF, it’s the most valuable plugin for WP development, in my opinion. It’s solid and powerful.
Report
I tend to agree with your analysis. I wonder if there are actual numbers on WP demographics. I certainly know very few serious website owners who are, as you said, “one-user-is-admin” type users.
Report
This. Over and over. And what about the developer creating sites for their own business? I absolutely do not want to use others Javascript React code in blocks, just as I avoided plugins. I absolutely do not want to write blocks for my use. Waste of my time. I hate being forced to find an alternative, which only means more work.
Report
Comments are closed.