Composer: create a local package repository to improve speed
When you're developing a lot of symfony projects with a relatively large set of dependencies, sooner or later you'll get annoyed by performance issues. By default, composer uses packagist.org to get package metadata (what versions are available and where to get them from). All packages are part of packagist.org by default, but 9 times out of 10, you'll only need a fraction of that.
So, to gain performance, I decided to generate a local package repository which only contains the package info for the libraries I regulary use.
1. Gather package names
In my homedir, I searched for composer.lock files, and extracted all github.com repository names with the following script, which, of course, started out as a oneliner.
#!bash
# command you can use to extract github.com package names from composer.lock files.
find ~ -maxdepth 3 -name "composer.lock" -exec cat '{}' \; \
| grep github.com \
| grep -v 'api.github.com' \
| egrep '.git",$' \
| awk '{print $NF}' \
| sed 's!.\+github.com/\(.*\).git",!\1!g' \
| sort \
| uniq \
> packages.list
This yields a file which contains github repository names on github.com like this:
Seldaek/monolog
symfony/symfony
...
2. Generate satis.json
Based on this list of packages, which now contains pretty much all of the packages I regularly use, I created a PHP template script that generates a satis.json for me:
#!php
{
"name": "Github libraries",
"url": "http://internal-satis-url-here",
"homepage": "http://internal-satis-url-here",
"repositories": [
<?php
$i = 0;
foreach (file('php://stdin') as $package) {
if ($i ++ > 0) {
echo ", \n";
}
echo " ", json_encode(
array(
'type' => 'vcs',
'url' => sprintf('https://github.com/%s.git', trim($package))
)
);
}
?>
]
}
Calling this script with the generated packages.list as input will render the satis.json file I will use to generate the packages.json:
#!shell
php satis.json.php < ./packages.list > ./satis.json
Now, it's just a matter of generating the packages.json with satis:
#!shell
satis update ./satis.json .
and copying the rendered packages.json to some URL available to your machine, lets says http://satis.local/
3. Disable packagist by default
You can disable packagist and enable your local repository by default, by adding it to ~/.composer/config.json
:
#!json
{
"repositories": [
{
"type": "composer",
"url": "http://satis.local/"
},
{
"packagist": false
}
]
}
4. Automating
You can simply automate this build process with a Makefile or any other tool you prefer for build automation. Makefiles make sense as they are pretty good at handling source file dependencies:
install: packages.json
scp index.html packages.json host:/var/www/satis
clean:
rm packages.json satis.json
packages.json: satis.json
satis update satis.json .
satis.json: packages.list
php satis.json.php < packages.list > satis.json
The end result:
In a sample project I tested, a full composer update
went from 3 minutes to 18 seconds, which still leaves to be desired for, but is nonetheless a 1000% performance improvement. Worth the try, if you ask me!
Happy composing!