If this were a feature project, how would you take a design-first approach with it? Also, given this original problem statement: When customers use Offload S3 along with another plugin that relies on and bundles the AWS SDK, unless that plugin is using v2, we encounter an issue of incompatible libraries. WordPress loads plugins in the order they were activated, and PHP loads files as they are required and classes as they are called (when using autoloaders), so it is a real lottery which plugin calls the SDK code first. The other plugin is the one that suffers. What's the ideal outcome / user experience for the end user? In what ways would dependency management improve the user experience?
i think the user experience of things mysteriously (mysterious to a non-dev regular plugin user) is pretty poor. Dependency management should allow things to simply work without conflict and that does improve the UX significanlty
The ideal outcome is not having the conflict in the first place (see section 4). The next best thing is having WordPress tell the user there is a conflict between plugins upfront. This is more ideal than the current situation where a plugin will just break, error, white screen, not work, or silently fail and the user is completely in the dark about what is wrong. Resulting in a trip to the support channel of one or both plugins, if the user identifies the cause themselves.
I think the UX would be simply raising a message on the install page that these two plugins are incompatible with each other. The user doesn't need to know the exact library and version conflict. The alternative is them realizing there is a problem when they experience a fatal error, or if they are lucky one of the two plugins blaming the other one for a conflict.
This has been discussed a number of times in Trac tickets already. Not being able to install a plugin because it has some invisible conflict with another plugin, is not acceptable for the core team (anyone got a recent comment link for this?). The end user is never going to be held accountable for any incompatibilities between plugins, within the WordPress ecosystem. That's just never going to change. So, I set on a quest to find the way to solve this, by making it a problem for the developers to fix. Sandboxing/prefixing turned out to be the least bad problem and so Mozart was born.
Yeah, I've seen those comments multiple times, I just disagree with them. Plugin conflicts are so prevalent even without including issues due to version conflicting libraries. They just tend to be hidden a little better than fatal errors. Saying that we should minimize the conflicts is an admirable goal, and prefixing is probably the only good way to do it, but that shouldn't stop the simple and straightforward solution being implemented first since it will be needed anyways as not all packages will be able to be prefixed. Why can't this be done iteratively?
But doesn't that already happens? If PluginB conflicts with PluginA and generates a fatal error, the plugin activation will fail. (Ignoring the silent/delayed conflict issues for now) Surely dependency conflict message would be a lot more user friendly than "A fatal error occurred"?
The library conflict won't always result in a fatal on activation. Plugin B might simply not work as intended as it is using the wrong library.
This has been a big issue for me in my development. There are a couple of libraries that are key for API connections that would take so much effort to recreate for a simple plugin that I want to provide to the community. I don't want to cause issues for other plugins, but I also haven't found a good alternative for dealing with dependencies. I'll keep an eye on your posts and look forward to any resolutions you come across!
Thanks Caleb!
I know this is only a small part of the deep integration you're talking about, but we've had great results using composer with johnpbloch's wordpress fork and wpackagist. They are at the core of the Roots.io wordpress stack, and go a long way towards alleviating the problem (at least for developers). Ironically, the only plugins we can't manage with composer are premium ones like WP S3 Offload and DB Migrate Pro :D
Using Composer to manage your whole site (WP, plugins and themes), like Roots/Bedrock does is great but it doesn't solve the issue here. If, with Composer, you require plugin A which bundles library X v1.0, and you also require plugin B which bundles library X v2.0, depending on how those plugins work with the different library versions can result in the conflict described in the post. Also our premium plugins are Composer friendly :) https://deliciousbrains.com/wp-migrate-db-pro/doc/installing-via-composer/
Well, this just made my day - I'm off to update my composer files :) Thanks!
Also recently switched to Bedrock. The approach of the boilerplate is amazing but it does not solve the issue of premium plugins, which is a whole other kind of issue, as Iain just pointed out. To solve the problem, you mentioned, Ed, we basically deployed a Satis (https://github.com/composer/satis) instance where we add the Deliciousbrains (and WPML) download links as suggested in the post Iain just pointed out. So, in our project we just have to add our Satis repo to our repositories. This way, premium plugins are managed centrally in our Satis instance. To make things even simpler, we wrote a script to parse a easy-to-maintain yaml file into satis.json (JSON is nice, but tedious to write). It also scans a dist directory on the server and inserts previously uploaded zip packages automatically into the satis.json. We use this for premium plugins which cannot be added by URL (such as WooCommerce Add-ons at the time of writing). Would love to see premium plugin providers like Deliciousbrains host their own Composer Repo (accessible maybe by access token) to host premium plugins, so integrating them into composer managed WP installs would be a breeze - and older versions could also be depended on. At least until a real solution comes along to manage premium plugins through composer. A big thanks to Deliciousbrains for for offering the end point, though, WooCommerce doesn't even have it anywhere in the roadmap as their support assured me. Linking their download links directly in composer.json leads to broken archives which do not unzip.
I've been thinking along similar lines for a while now. I'll give Satis a go - thanks for the suggestion :)
We certainly have it on our radar to improve our Composer support, specifically to allow specific version downloads, but it's not high up the priority list :)
Awesome! Looking forward to hearing about it sometime in the future.
Yes Premium plugins can be a RPITA when it comes to composer updates. What we have done at my company we have a private git repository for each premium plugin. We are using composer to require those into the appropriate site, but it is a lot of DevOps management on the initial as we have to make sure we properly tag each version correctly and it's more to maintain. Take everyone's favorite SEO plugin. Every time there's an update we must request the lasted download link. Then unpack it, commit it to our repo and tag it. It's a few hours of turn around but since we have an unlimited license all of our sites receive the update from our private repo and we can push out the update relatively quickly form there. The are alternatives, but I think all get messy. As each premium provider has their own way of doing things. We've just ensured it's standardize at our edge. Cheers, m
Your Satis link is borked. It lands in GitHub 404 because the paren is being appended. https://github.com/composer/satis
Another solution for managing premium plugins is https://github.com/blazersix/satispress . It has the advantage that you can make use of the standard WordPress update mechnism to load new versions.
Good shout, Alain - I use that myself :)
Great hint, how did I miss this?! I assume one sets up a Wordpress instance just to expose the packages, right? Only drawback would be single-install licences I guess.
Doesn't roots.io already deal with this?
Roots allows you to manage your whole site with Composer, but the problem can still occur. See http://disq.us/p/1h7b5gg :)
With Bedrock? Iain is referring to dependencies of plugins, not of entire WP projects (which bedrock kinda solves).
i would be extremely happy to see something composer based in the core we were able to solve the similar jQuery issue years ago and now it's time to solve it with generic dependencies
Great article. And it is only one weak part of the problem: think about the numerous JS libs called, sometimes twice with different versions and, as a user, I am horrified that I can not master what my website is calling outside its domain.
Ugh. I am one of the vocal opponents of including Composer in WordPress. I won't repeat my objections here because they can easily be found in Trac but I will leave this ditty: "A WordPress developer had a problem managing dependencies and thought 'Wow, I can use Composer to solve that problem!' Now the developer has two problems."
BTW, I addressed the dependency problem in Imperative back in 2012/13. But I really needed some changes in core to make it work flawlessly and I could get no interest from the powers that be to see those changes in core. So I gave up on the issue and focused my business away from building plugins for clients for distribution. Imperative addresses the first point from Ryan McCue: "Catch potential conflicts with dependencies up front before a plugin is installed."That said, with changes something Imperative-like would be a much easier for people in the WordPress world and would not require all the hackery that is required to get Composer to accept and allow the WordPress defaults, nor would it require all WordPress plugins to go back and implement a composer.json file (yet another flawed aspect of Composer, IMO.)
Wouldn't plugins need to retrofit to the Imperative method?
Mike, I know you are an opponent of Composer in WordPress, but let me repeat the most important quote from this article, in this whole 'why use Composer'-debate: > The problem is already there, without Composer, you just don’t notice it as prominently. Packaging any third-party library with your public plugins, whether you use Composer or not, will cause issues with WordPress. It's not a problem with Composer.
rarst has a good page regarding composer in wordpress http://composer.rarst.net/
See https://github.com/mzalewski/wppm It bundles Composer and uses Composer's dependency solver to resolve plugins conflicts (or notify the user that PluginA cannot be activated). My question is: Do we want to allow users to automatically resolve conflicts by downloading different dependency versions? Composer is a dev tool, not a user tool. While I definitely think WordPress needs to add some kind of dependency management to handle conflicts, I don't think it should be allowed to modify the WP installation and potentially introduce some site-breaking changes that could be a nightmare to debug.
Thanks for the link, that looks like an awesome piece of work - I wish I had seen it sooner! Different dependency versions can't be loaded together without some form of prefixing, and if that were possible I don't think the user needs to know that is what is happening, WordPress just needs to do it without worrying the user.
This is a great well written article. Like many of the commenters below, we have been using Bedrock from roots.io for our base install. I recently had the need to integrate a Laravel app with a Wordpress backend. Using the s3 drivers from League's Flysystem for Laravel 5 requires AWS SDK v3, and the delicious brains plugins use v2 - basically an unsolvable dependency conflict! It's time Wordpress catered more for developers by integrating with the tried and tested industry standard solutions out there like Composer.
Iain, I feel that this is a big problem for WordPress in that all of the other serious PHP based projects are composer friendly and we've all just kind of done our own thing. I started down this road (see: https://github.com/mikelking/dummy). In any event I got dummy working to the point of building a working site out of several different layers of composer abstraction. Ultimately getting hung up on some of composer's 'There can be only one' obliterate the destination directory magick and making certain it works in ALL dev environments. Keep in mind that .org has abandoned PHP5 altogether (as an end of life prospect). See https://wordpress.org/about/requirements/ which stats the following: * PHP version 7 or greater * MySQL version 5.6 or greater OR MariaDB version 10.0 or greater * HTTPS support I don't think that you need to get .org onboard w/ composer out of the gate. My vision for the dummy project was to make it easy for a developer to fork dummy and then define their dependencies in the composer manifest and build their plugin or theme without having to worry about the other details. The secondary goal was to make it possible for anyone to to fork dummy to easily define a new site in a consistent manner using the composer manifest and then use tools like jenkins or deploybot to ship the code to their hosting provider. In both cases the composer magick happens before the code hits the web heads. I think there are some synergies to what you want to do and what I've already done, thus if you want to discuss hit me on twitter: @mikelking:disqus
Hey Mike, thanks - I will take a look at your project :) Unfortunately, WordPress haven't abandoned PHP5 altogether, they still support backwards compatibility to 5.2 but 'advise' PHP 7 should be used.
Good article, and I would love to see Composer in WP, as well as contribute to the development of the associated software, etc. If someone takes this on, feel free to get in touch with me. There are however a few points I would like to make: 1. Mozart is not a good solution. The namespace is part of a class' FQN, the whole point of which is to be able to identify a class uniquely. PSR-4 makes this possible by requiring a vendor/package prefix. Of course, it is possible for 2 vendors to have the same name, and for them to have 2 projects with the same name also. However, this is quite unlikely, and if this happens it can easily be fixed. Therefore, I don't see any problem here. And no need to screw with other people's code. Leave the namespaces as is, and just use PSR-4. Clever people have already created all the standards and tools we need to make this possible. It's just that WordPress developers are so attached to usability, that they tend to forget about practicality. In this case, trying to make any combination of plugins possible is something that is unnecessary. There is such a thing as incompatibility, and it's alright. It's there for a reason. 2. No need to "heavily customize" anything. Of course, there is need for some customization. The composer/installers
package works great for these things. oomphinc/composer-installers-extender
can be used to avoid having to write an installer, if needs be, even if temporarily. wikimedia/composer-merge-plugin
can allow us to merge configs from multiple locations, which looks like a good way to solve some of the problems. And we'll probably end up writing a Composer plugin, yes. But this is not necessarily huge. Many problems can be solved via convention. On the other hand, I understand that this is probably a case of just abusing the word "heavily". 3. Plugins can also be required. There's no problem with requiring other plugins, if they provide something useful. A plugin is just a package. However, it would be much better if plugin dependencies were libraries instead. And for this, developers need to write better code. So what? Many people have been writing good code for like ever. If some people can't, that's OK. Maybe, it's even impossible to provide a way for consistent management of plugins via Composer. Maybe, that's OK too. As long as it's possible to manage plugin dependencies through WordPress with Composer. 4. Develop the functionality standalone. As Carl Alexander says, you've got to have really thick skin to be able to stand against the human factor when contributing to WP, and I am not willing to put myself through that torture. Instead, just develop a standalone tool, and wrap it in a WordPress plugin. If WP Core wants to integrate it, they can. If not, well, what's the problem? The tool would be available with an open-source GPL-compatible license such as MIT, and those who want to use it will use it. Which I'm guessing will be a great many. Anyhow, like I said, I'm available to help out here, whether it's hands-on dev work, or planning, project management, whatever. You can check out my work at RebelCode and Dhii to learn about the way I do things.
Hey Xedin, Regarding your first point, I think you misunderstood what Mozart tries to achieve. This is not about making two separate packages work that have conflicting namespaces. This is about dealing with the fact that WordPress does not have one central location to use Composer. So, if you use Composer at the plugin-level, it is used several times in isolation, and cannot achieve its main mission of only loading 1 version of a given dependency used across all plugins, because it doesn't know about the other plugins. This causes libraries to be included several times, once for each plugin, and potentially in different, conflicting versions. As the autoloader still only loads only one single version, the actual version you get is very random and will very probably lead to bugs. Mozart tries to solve this by prefixing each package, and thus loading all of them, one for every plugin that uses it, so that every plugin gets the version it needs.
Hi Alain, This is what I meant, since it is effectively the same things. Two packages that have colliding namespaces will work together, because the namespaces are prefixed by Mozart. I believe my point still applies. And yes, if we use Composer for each plugin separately, this is a problem somewhat. But we are talking about installation-wide management, aren't we? This problem should go away on its own.
Yes, installation-wide management would be the ideal outcome, indeed. Mozart is more of a plan B if all efforts to introduce a centralized Composer fail. Mozart tries to make Composer at the plugin-level work across multiple plugins.
Disqus marked it as it contained a link, approved :)
Thanks! :D
This is a great topic and I would love to help see this happen. I'll do what I can to support it as a project, but if my efforts with contributing to WP-CLI are anything to go by, I'll mostly be logging issues as I find them in using the solution. I'm currently developing a plugin for a client that bundles the AWS SDK, I'd be interested to hear how you have worked around the problem in its current incarnation. Or if you get a support request do you just ask the conflicting plugin developer to update their SDK version?
Addition of composer in WP great..!!
I would love to see composer in WP
Automattic, the company who owns WordPress, DOESN'T WANT THIS FOR BUSINESS REASONS. It's not a technical issue, it's a business GOAL for them not to have a dependency manager like Composer. If you integrated Composer into WordPress, you would then break the gatekeeper function of the Automatic company. It would essentially smash the .org plugin repo [as you could now pull source from anywhere]. THIS WILL NEVER HAPPEN WITHOUT A HARD FORK. This is a great discussion, but you guys all miss the point. AUTOMATTIC IS NOT YOUR FRIEND. There already is 100% dependency management the way Automattic wants it. You give a notice to the user that they need to download another plugin from the .org repo OR YOU DON'T USE THE DEPENDENCY. That's how they want it!