Add REST API endpoints for widgets and sidebars

[westonruter]

There is currently no proper client-side API for managing widgets and the sidebars they belong in (and any APIs in PHP are also lacking, per #16613). Any widget manipulation requires a lot of manual Ajax calls and reverse engineering of how the widgets admin UI manages widgets.

The implementation of normalized REST API endpoints could coincide with re-architecture of how widgets are stored by making use of a custom post type instead of storing widgets in options. See #35669.

The way that widgets and sidebars are represented needs to be formalized and normalized in REST API endpoints (ideally in conjunction with nav menus).

Potentially any endpoints for widgets could be abstracted away as “content blocks”, such as with #33473 (Shortcodes + Widgets + Nav Menus. Unified "component" API (aka Content Blocks))

See discussion at ​https://github.com/WP-API/WP-API/issues/19

(Originally this ticket also described a JS API in front of any Ajax/REST interface to facilitate the manipulation of widgets and sidebars, but the scope is now limited specifically to the REST API and any JS interface can be part of the WP-API Backbone JS Client.)

[DrewAPicture]

The widgets admin UI needs a rewrite to utilize Backbone.

What justification is there for rewriting this workflow other than just to use Backbone? Are there user tests showing that this workflow needs to be rethought? Is the workflow broken?

Or is this simply rewriting to rewrite with Backbone? If it's the latter, I don't think that's ample justification.

Don't get me wrong, I'm all in favor of kaizen when it comes to our code base, I'd just like to hear more about why a complete overhaul is necessary and justified.

[westonruter]

Replying to DrewAPicture:

The widgets admin UI needs a rewrite to utilize Backbone.

What justification is there for rewriting this workflow other than just to use Backbone? Are there user tests showing that this workflow needs to be rethought? Is the workflow broken?

Or is this simply rewriting to rewrite with Backbone? If it's the latter, I don't think that's ample justification.

Don't get me wrong, I'm all in favor of kaizen when it comes to our code base, I'd just like to hear more about why a complete overhaul is necessary and justified.

Absolutely. Yeah, it's not just to rewrite for rewrites sake. There are a few reasons for doing this, which I need to flesh out more fully (and will need to be done in the description here).

For example: The addition of the widget-added and widget-updated hooks (#19675, #27491) required them to be added to be three separate areas of the codebase because there is duplicated logic, making maintenance a headache. Introducing widget management in other areas, for example in a TinyMCE view or in a frontend editor, would require yet more duplications of the widget control form logic.

Another example: Currently when a widget gets added to a widget area, a hidden "template" widget control element gets cloned and then injected into the widget area. This causes all kinds of problems with event listeners and dynamically-built controls (e.g. via jQuery UI) since the cloned element don't come with the template's event listeners and bound data. Widgets controls should instead follow the Backbone model of having an underlying template string which gets constructed into a view.

There is no JS API (or PHP API for that matter) for manipulating widgets. Using Backbone for a standard JS Widget API would allow core (and plugins) to have a unified interface for managing widgets, and allow them to be easily used in many more contexts. Adding widgets to the customizer would have been much easier if there was such a Backbone API for managing widgets.

[kadamwhite]

If I can ask a question to clarify the intent here: it sounds like what is being proposed is (1) a Backbone re-write of the widget management UI, and (2) the addition of some endpoints for getting widget information via AJAX. The former seems to be the goal here, making it easier to add widgets to the customizer, etc, but this would require the latter in order to get and update information appropriately, correct? If so, I think it's important to be explicit about both components of the problem—the idea of a widget endpoint is also an open to-do for the json api project, ​https://github.com/WP-API/WP-API/issues/19, and in that project Ryan calls out that a widget endpoint also makes assumptions about a sidebar endpoint, sidebars relationships to themes, etc.

I'd like to see this happen, I just want to make sure the scope of the project is clear.

[chriscct7]

@westonruter see comment:6

[wonderboymusic]

This ticket says way too much and talks about JavaScript in the same breath as proposing REST API endpoints that don't exist yet. Do we want to clean up the description to just focus on the endpoints? What is built with the endpoints is irrelevant.

I wrote a woefully insufficient, read-only endpoint for Sidebars here: ​https://github.com/staylor/graphql-wordpress/blob/master/packages/graphql-wordpress-middleware/lib/REST/Controller/Sidebars.php

The data model for widgets is overdue for a spec.

Widgets on their own in a REST API seem semi-useless. Sidebars are a mandatory requirement for building a frontend that reflects the data in the admin.

Can we use this ticket as a spike for the endpoints? Everything else is separate.

[westonruter]

Agreed. The scope of this ticket should be limited to REST API endpoints.

There are some prototype endpoints for widgets in the ​JS Widgets plugin. However, it depends on widgets implementing the JS_Widgets interface so that there is a schema available for each widget. See #35574. To me it seems for widget endpoints to really be usable in JS they should expose the instance props not just the PHP-rendered WP_Widget::widget() output. In this way the widgets could be rendered using JS instead (e.g. React). However, since schemas aren't available for each widget yet, we don't have a way to manage the visibility of the properties. We could just output the rendered widgets if they have no schema defined, but we should provide a schema in core to make that an option. Schemas have been implemented for media widget instances.

One other gotcha for widget endpoints: a widget may render differently depending on the page it appears. For example, a Text widget with a [gallery] shortcode in it will render differently depending on which post is the current queried object. Likewise, a Navigation Menu widget will apply different classes to menu items based on what the current URL is.

There is an existing WP-API GitHub repo for developing widget and menu endpoints here: ​https://github.com/WP-API/wp-api-menus-widgets-endpoints

Lastly, I haven't been wanting to push to hard on adding widget endpoints yet while the work on blocks is underway in Gutenberg. Blocks in Gutenberg actually have their own dedicated wp_block post type (which I've wanted to do for widgets in #35669), and the data model for blocks is being architected in the way that widgets should have been done in the first place. I personally think that widget endpoints in core will make the most sense after we go through the work of migrating from widgets to blocks in sidebars. The data model of widgets is a mess and trying to expose endpoints to manage the mess will be messy. Also, nav menu items are similar in concept to widgets (of a specific type) but their data model and API are wildly different. The endpoints for both widgets and nav menus should be closely aligned in interface, but that's painful now because they are so different architecturally.

So once we have blocks then I think we'd be in a better place to work on the endpoints for widgets and nav menus, because at that point we'll be working on migration anyway. It may end up being that we actually don't implement endpoints for WP_Widget at all, but rather only endpoints for widget blocks. Since all widgets will be ported over to blocks, the need to interface with classic WP_Widget will lessen and eventually could be deprecated altogether.

[wonderboymusic]

Blocks (to me) seem like a UI concept divorced from the notion of core models. Blocks seem like an interface containing types, options, reference(s), and content (also known as a parsed Shortcode).

Widgets currently have a crappy interface (data, not UI), but those details could be hidden so the interface survives a migration.

interface SidebarInterface {
  public function get_options(): array;
  public function get_widgets(): array<Widget>;
}

interface WidgetInterface {
  public function get_options(): array;
}

class Widget implements WidgetInterface {
  // existing bad fields that may change
  private $a;
  private $b;
  private $c;
  private $d;

  public function get_options() {
    return [...whatever from wherever...];
  }
}

I think we could create a "legacy" wp/v2 endpoint and then bump the version to wp/v3 for what Gutenberg wants to berth using the same interfaces.

[westonruter]

Blocks do have a UI component, but they also have a schema. A block has attributes/properties just as a widget instance does. A block is richer in that it can define sources for these attributes outside of just a JSON object but rather can refer to data in HTML, postmeta or eventually other sources as well. Widgets indeed do have a crappy storage mechanism in serialized options with an equally crappy way of identifying a widget with via a {type}_{number} “ID” (like text-123) which is totally different from how everything else in WordPress is identified: creating a new widget instance ID relies on grabbing the max index of all current array indexes for a given type and then adding one; this makes them very prone to concurrency collisions. Widgets need to move to using auto-incremented IDs just as posts use… or better, a UUID like Customizer changesets use.

The wp_block post type in Gutenberg uses UUIDs for IDs and this is how they are accessed via the blocks endpoint:
​https://github.com/WordPress/gutenberg/blob/13c7c99e17fe8c6b1bee78d43a24e1e805b723c4/lib/class-wp-rest-reusable-blocks-controller.php#L282-L316

Nested blocks are on the horizon for Gutenberg, and this is essentially what a widget sidebar or nav menu can be considered: reusable blocks that contain an ordered list of widgets or nav menu items.

[peterwilsoncc]

An API endpoint was added in #41683.

I'll close this as a duplicate even though this is the earlier ticket.