Fields API: Request for review of direction
Over the past many months this year, I have been working with guidance from @helen on the new Fields API with the intention of inclusion into WordPress core. It’s based on the Customizer API, so those who have experience with it will see a lot of similarities. The goal isn’t to create a duplicate API though, and in our proposal we would be implementing the Fields API within the Customizer API.
What does that bring to core as a whole? It gives core a very real and far reaching API to add fields to any type of object in a standard interface. Given the growth of the Customizer API and its inclusion of fields, @helen and I both agreed it’d be great to leverage everything it can do in our efforts here. The goal isn’t focused on the actual Customizer UI or editing on the front-end, it’s all data-oriented.
We would create implementations behind the scenes in the various existing APIs such as the Customizer API (now) and Settings API (later) while maintaining backwards compatibility. We’re also hoping to implement the Fields API within the User profile screen. This would give core more flexibility towards revamping the User profile screen in the future, without having to tackle even more than just a UI refresh. That’s also not mentioning the fact that plugin and theme authors could leverage this API to extend WordPress with their own fields in the various areas. Future places we’d look at integrating Fields API with would be the Post editor, which I’ve pegged as our third focus, behind User profile fields and the Customizer API.
Anyways, that leads us to the point we’re at now, we have an API but we don’t have all of the built-in Field Types (control types) yet or the latest Customizer API changes from 4.3 merged upstream into the Fields API. There are unit tests that are running (and passing!) so that is going to help us on our road towards core for sure.
We need developers to give their feedback on the direction we’re heading. Here are the relevant links for those reviewing:
- GitHub
- Fields API Specifications (so far)
I’d love more feedback from anyone contributing to WordPress, or the *countless* plugin and theme developers who would use the Fields API. Please comment on this post, or if you’d like to discuss further, you can hop into our weekly meetings on Mondays (Monday 20:00 UTC 2015) in the WP Slack #core-fields channel, or ping me there anytime.
Update – 08/10/2015:
We now also have a growing list of developers who are behind this project and would implement it in their projects if it were merged into core.
Matt Gibbs 4:48 pm on August 3, 2015 Permalink | Log in to Reply
Thanks for all the hard work, Scott and Helen! The specifications doc looks very detailed. I’d like to see more usage examples though. Similar to the one on https://make.wordpress.org/core/2015/05/27/metadata-api-project-reborn-the-new-fields-api-project/
1. How to create custom field types (or Controls as its called now)?
2. Does a field require a section?
3. What happens if the field capability isn’t set? Does it fallback to the parent section / screen?
4. How are repeatable fields handled in the current spec?
5. Is the field API itself concerned about field order? Should it be?
6. How granular are field sections? E.g. can a field be added to a single post item (or user) only?
Scott Kingsley Clark 5:55 pm on August 3, 2015 Permalink | Log in to Reply
1. We’re still ironing that part out, control types / field types is the next area of the spec we have to nail down. Feel free to join us on Mondays for our weekly meetings if you want to help us get that part finished.
2. Currently, it functions the same as the Customizer API, which allows you to add as many fields as you want, without sections or screens (panels). The differentiation is when you add a control, you specify the section or screen you want to add it to.
3. Fields don’t require capabilities, controls don’t either. We handle the same capabilities checks as the Customizer API does right now — https://core.trac.wordpress.org/browser/trunk/src/wp-includes/class-wp-customize-setting.php#L439 — which is to say, if you don’t provide a capability check, it is wide open. We should nail this down further and I think it’d be good to default to no-access. We need dev input on that.
4. There is no preparations for repeatable fields or repeatable groups at the moment, that’s something we want to nail down in the control types / field types discussion, as there may be things we want to do there. The existing Fields API spec is almost entirely based off of the Customizer API and no additional functionality has been added for that yet. It’s on our list though, we want to ensure any effort we put forth into core does not limit developers or the common ‘loop field’ use-cases. Whether ‘loop fields’ go into core or not, that’s not something I can speak to at this point.
5. Yes, the spec makes use of the ‘priority’ option for screens, sections, and controls. Works just like filter/action priorities do.
6. Screens, sections, controls, and fields are all added to a specific object type / name (for example: post type : post). Further definition based on a conditional like a specific ID, capability, etc, could all be handled through an additional callback or filter that we can add inside of the check_capability methods for screens / sections / fields.
Brent Jett 8:23 pm on August 3, 2015 Permalink | Log in to Reply
Cool spec! I like the idea of a broad-reaching common syntax for adding fields from themes/plugins, but I’m wondering if there isn’t a more modern/human-readable format or syntax for this. My immediate thought was JSON files (OOP code underneath obviously). WordPress exposed object oriented API so late in it’s relative lifespan that OOP code like the customizer API still looks very foreign to themers. Given that most app platforms now use JSON manifest files to declare things like configuration, it would seem to me very natural for WP to look at adding a more approachable syntax like that to it’s theme configuration APIs.
But I could be way off. Nice work either way.
Scott Kingsley Clark 8:41 pm on August 3, 2015 Permalink | Log in to Reply
That sounds like a pretty cool idea, it could wrap the Fields API itself. It would be better put with a similar JSON handler for registering post types / taxonomies too, I’d think. Something wider-reaching than just Fields API.
Brent Jett 8:52 pm on August 3, 2015 Permalink | Log in to Reply
Totally agree. I’ve got a working plugin for a generalized theme config API using JSON to allow things like theme supports and enqueuing stylesheets/scripts from a config file. In practice it makes it way easier to see how a theme is set up and it prevents goofey php errors like missing semicolons. There’s really no need for configuration to be done in a programming language. If you’re interested, i’d be happy to send you the repo.
Brent Jett 1:01 am on August 4, 2015 Permalink | Log in to Reply
I got the bug to see what your examples might look like in JSON. It’s not bad.
https://github.com/brentjett/wp-theme-configuration/blob/master/docs/spec.md#proposed-fields-api-experimental
Paal Joachim Romdahl 1:11 pm on August 4, 2015 Permalink | Log in to Reply
This looks really interesting Brent!
Scott Kingsley Clark 4:05 pm on August 4, 2015 Permalink | Log in to Reply
Very cool, this is a great example of what the Fields API itself could enable!
Mike Nelson 10:14 pm on August 3, 2015 Permalink | Log in to Reply
In Event Espresso we created a similar system called the “Forms System”, inspired primarily by Django’s forms. EE Forms basically define how to display an HTML form, how to validate the user-provided input on both the client-and-server side, and how to sanitize/normalize the data for use in the PHP code. (It leaves the job of saving data to either our models system, wp options api, and sometimes direct wpdb).
EE “form sections” are roughly equivalent to your “screens” and “sections”, and EE “inputs” are roughly equivalent to your “controls”.
Some differences that might spark some ideas for you:
Other than that, it’s really quite similar.
Also it’s not that clear: is the fields API intended for use in the frontend too?
Also it seems if this is intended primarily for defining the HTML forms and how they get validated/sanitized. I think having a separate API for defining the underlying data would be beneficial, in cases where you don’t want every attribute of the data to necessarily be represented in HTML forms, or you might want to provide different forms for editing the same underlying data.
Scott Kingsley Clark 4:16 pm on August 4, 2015 Permalink | Log in to Reply
1. I’d like to explore the idea of sub-sections and sub-fields (sections that can have sections in them, fields that can have fields in them) in our class architecture. Even if they’re not utilized in core, the ability to use them or for plugins to enable their use would be a big thing.
2. We’re looking at something similar, though not really, it’s mainly that each screen will have it’s own fields display and saving implementation, much like the Customizer itself has one.
3. There are filters and actions in place for this right now in the Customizer and the Fields API
4. I don’t doubt someone could extend the Fields API with a new control type or a custom section class that would be able to output just HTML as set in the definition options. That could be useful for some cases.
5. Fields API is based on the Customizer API, it’s somewhat split up between modeling and display, but it’s not entirely MVC at the moment.
6. You can use the Fields API anywhere, you would write your own implementation to display fields and process the $_POST data accordingly, utilizing the underlying Fields API to access the data structures and field types input HTML etc. You could even use the Fields API for any object type you want to build into it, there are hooks in place that support custom object types, for purposes of plugin developers to utilize the Fields API for their own things (Gravity Forms, Pods, NextGen Galleries, etc).
7. Each implementation can override the core output, to do whatever it wishes. For instance, Customizer API supports previews and makes heavy use of JS to do what it does. Our Customizer API implementation extends the Fields API but does not limit the ability for those things, and many others for other implementation needs.
thomask 11:02 pm on August 3, 2015 Permalink | Log in to Reply
I am really looking forwards to this, actually for me this is the second most crucial missing feature for WP – the first is multilanguage support. I use Advanced Custom Fields for almost every web i do, and however it is very nicely written and easy to use, i would prefer some build-in support.
I have read it all just briefly and will have to try it, from my experience what is important is
1. support for multilanguage fields (so that i can have common field for all languages version of one post, as well as translatable field) – but this is i guess more question on the multilanguage plugin (or future WP multilanguage support)
2. text field (just simple text), term fields and post field (N:N relation field), wysiwyg field, user selection field … and/or easy way how to create new field types with standardised look, values control etc.
3. allowed values control and definition, read only possibility, required field possibility
4. possibility to add fields also to terms (!!!) – imo terms (edit-tags.php) are currently very stupidly different from the posts – it should have the same look, so that we could use the same fields and functions.
5. easy definition of post types and other conditions, that are required for field to be displayed
6. advanced feature, but very usefull in many cases – possibility to set-up condition for field, that should be calculated live on the post screen depending on some live changes – e.g. “this field should be visible only when category is “blabla”).
7. field could be added also to attachments – but dont forget, that it does not have only the standard post.php, but that the fields are displayed also on the attachments popup screen
8. dont forget the field versioning (with older version of post you see also older version of its fields)
9. posibility to create groups of the fields with their conditions
10. possibility to e.g. display fields in two columns
11. very advanced field – repeater field (e.g. something like this http://www.advancedcustomfields.com/add-ons/repeater-field/) – possibility to create field table. This is very complex, but very often this is the only way how some things can be solved, as currently there is only simple ID->key->value in WP, but very often you need ID->key->array(subkeyN->valueN), so without this field type many users will still have to use some solution as advanced custom fields.
Scott Kingsley Clark 5:26 pm on August 4, 2015 Permalink | Log in to Reply
1. That sounds great, it sounds like Fields API could enable this through the options you pass into the configurations. A plugin could pick that up and change how the fields work from there, per language.
2. That’s on the list at the moment
3. Good ideas, we’re still nailing down the specs but those are two I’d like to see us tackle if possible — it’s just going to be difficult.
4. Not currently possible, but when Term meta makes it’s way into core, this Fields API will be there to utilize
5. Sounds great, I’m not sure how much of this we’ll tackle in the Fields API itself, but it’ll definitely be doable by a plugin.
6. That also sounds great, plugin material though — but enabled by our extensible code.
7. This sounds great too, it’s not part of our initial focus but there are hooks in place that we could implement this for media fields as well.
8. This isn’t a problem, everything operates by ID, revisions get their own ID, so everything should continue to work there, even when post meta revisions makes it’s way into core.
9. Groups of fields are possible, but conditions is currently plugin territory.
10. That sounds like plugin territory, but totally possible because we have hooks for that.
11. I love the idea of a loop field or some basic repeatable field in core, not sure if we’ll include the field type itself in core but we’ll do everything in our power to enable developers to make use of it in the numerous plugins that have this field type as an option.
camu 1:16 pm on August 4, 2015 Permalink | Log in to Reply
Thank you for all this work. I was wondering, why not just merge Advanced Custom Fields into core? It’s a robust plugin that already does most of the things listed here above. I’ve been using it for years now, and it has always delivered.
Scott Kingsley Clark 4:04 pm on August 4, 2015 Permalink | Log in to Reply
There aren’t any plugins that could be a candidate to merge into WordPress core right now as they are, this Fields API effort is our focus right now to make a better API for custom fields plugins out there to utilize for a better standard when developing for WordPress. What we’re doing here wouldn’t replace custom fields plugins for many developers, they add features that are useful for some projects and offer an admin UI to manage these things. At this point, the focus is not on creating an admin UI to manage custom fields for objects in WordPress, but to offer up a unified API to interact with fields for all object types.
khromov 4:53 pm on August 4, 2015 Permalink | Log in to Reply
From reading the post, the proposed direction is very similar to what ACF offers. There will be edge cases that the new API won’t fill right away (ie. repeatable fields), but it’s something the community is going to want and build.
My point is, don’t understimate the will of the community to bend this to their particular use case. A good design (which it looks like so far), enables that.
Scott Kingsley Clark 4:57 pm on August 4, 2015 Permalink | Log in to Reply
Totally agree, this is actually the result of lots of communication between many developers of plugins / libraries for building custom fields. There’s no way I would underestimate this community, was just noting that none of the plugins were a direct fit.
christopherabalos 7:49 pm on August 4, 2015 Permalink | Log in to Reply
Very interesting take on a much needed API. The WordPress API for creating metaboxes, post types, taxonomies, customizer fields, widgets, etc are in need of a uniform API. I’ve been attempting to tackle this the last few months in my own projects. My implementation revolves around new manager classes for things like post types and settings that follow a pattern similar to the Customizer API. My manager classes have been written to accept WP_Customize_Control objects.
I’d also like to see the ability to create fields for widgets, which is something I’ve included in my implementation.
I’m curious to hear more about the decision to wrap the fields API into a single manager vs. having separate managers that all accept the same field objects.
Keep up the great work Scott!
Scott Kingsley Clark 3:21 pm on August 5, 2015 Permalink | Log in to Reply
Widgets is totally a logical next step in this, I would love to see Fields API used for Widgets, for the Shortcode UI project, etc.
Scott Kingsley Clark 3:24 pm on August 5, 2015 Permalink | Log in to Reply
One manager class is based on the Customizer API structure. I ended up going with one class so that there was one class things would be used to interact with, stored screens/sections/fields/controls in, etc.
Chris Lema 3:43 am on August 5, 2015 Permalink | Log in to Reply
My question may be silly, but assuming that a field has been created and attached to a post type, and then populated for all those posts, is there an expected facility in the API to be able to return a list of posts where the field is populated with a specific value? Additionally, are their collection operations where I could get a count of all those posts with the value of a specific field matching what I’m submitting?
Scott Kingsley Clark 3:20 pm on August 5, 2015 Permalink | Log in to Reply
WP_Query / meta_query already do this
The Fields API is not meant for querying for data, it’s to register the structures of information a site needs for it’s content / data types.
jbmanos 11:38 pm on August 7, 2015 Permalink | Log in to Reply
So if I’m understanding this correctly, I could register fields I’ve developed for a particular site and the core will know what fields it needs to show the data it has? In other words, this would be a manifest of information types available?
If so, I like it! I already get a little nervous that if I lose a piece of php that registered the fields to begin with, how many SQL queries and guess it’ll take to recover the data.
Scott Kingsley Clark 6:50 am on August 10, 2015 Permalink | Log in to Reply
Basically, yes. A standardized API to register fields for different objects. An API that would eventually be the under pinning of all other Form UI, in that — a plugin, theme, or app (as exposed by a REST API) could know everything about your WP site, what content types it has (already possible), down to what fields it has (already possible) but more importantly — what types of fields it has. More in-depth information about the fields, where they are, how they’re used.
If you lose the PHP side of this, you’re back to where you were before, which is to say, the same place you’d find yourself if you lost the config in any other plugin that builds fields. So there’s not really anything to get around that
Paul de Wouters 6:35 am on August 5, 2015 Permalink | Log in to Reply
Not sure if it has been mentioned, but custom fields for menu items would be useful too.
Scott Kingsley Clark 3:20 pm on August 5, 2015 Permalink | Log in to Reply
That’s a whole other story for another time, but I agree. Menu item custom fields are currently a very big affair to deal with given the lack of actions/filters in that area of the UI.
Eric Andrew Lewis 7:24 pm on August 14, 2015 Permalink | Log in to Reply
Defining a spec for this project is a great way to go.
The spec can be drawn out in some places. e.g. What is a field, and how does it relate to object metadata? How does this relate to prior HTML form HTTP POST submissions? This would help describe the problem space.
The cross-platform usage of self-describing fields/metadata for rendering custom views (e.g. rendering a color-picker in the iOS app, or rendering custom media fields in Attachment Details) is a major plus, which should be explained in Community Benefits.
The “Background” section (as-is) doesn’t seem relevant in spec, as it describes the history of the project, not specification.
Scott Kingsley Clark 7:44 pm on August 14, 2015 Permalink | Log in to Reply
Good points, thanks for that, I’ll be sure to address those in the spec next.