This post is inspired by a comment from CaptainHawk and by Linux Loop’s article “Repository Adding Via Apt-URL – An Overlooked Feature of Ubuntu 9.04“. So, let’s look how the feature will be implemented and why it is overlooked. (hint: because it sucks).
Introduction, apt-url and the problem (you can skip this)
Thanks to the .deb format, installing any software not present in the repositories is as easy as downloading the file, clicking on it and filling your password. If the program is present in the Ubuntu repositories, instead, thanks to the magic of apt-url, it’s possible to make up an url that asks Ubuntu to retrieve and install such software from the repositories you have enabled.
How to install LOLCATS on UbuntuThat’s quite nice, but still something is missing.
- installing from standalone .debs doesn’t get you the automatic system software update you’ve grown to love and respect.
- complex programs have dependencies. This means they require other software parts to work properly.
- If those dependencies are not included in your repositories you’re out of luck.
- You have to install them manually.
- Even in the case you can find them packaged, you have to install them in a specific order (for example look at pidgin on getdeb).
That’s the reason why most complex applications usually have their own repository. Once an user adds their repository to his list, he gets the very same features he get from official repositories. Automated software updates, installation via apt-get, installation via apt-url. And he doesn’t have to install the dependancies of the application manually, because those are (hopefully) included in the new repository.
Currently, though, adding a new repository is not easy: you have to manually tweak your repository sources list.
- You can do that in System->Administration->Software Sources, going to the second tab, and click the Add button. Then you copy paste there the code you found on some website. Usually repositories instruction consist of two lines to be added (the second being the source code repository, which is unneeded in most cases, but a novice has no way to know that), so you have to repeat the process for the second line.
- Or you can add them manually to the /etc/apt/sources.lists file. So you browse with nautilus to the /etc/apt directory and open the sources.list file. You paste the two lines, save and.. ouch, can’t save, it’s readonly. Yes, gedit won’t ask you for a password if the file is readonly it will just refuse to write it. You can workaround this opening a terminal and issuing a command as unmeaningful as gksudo gedit /etc/apt/sources.lists. Once you’re there add your repository code at the end of the file, be sure to not have changed anything other, save and quit.
Then force a sources update by issuing: sudo apt-get update.
Now you have the new repository enabled, and you can install software from that in the usually ways. But you’ll get horrible errors when updating and a further confirmation dialog when installing because the software you’re installing is not verified/signed/authenticated/whatever.
What you need to get rid of all that uglyness is to import the keyfile for the repository. This means downloading it from the website and importing it from the Software Source dialog or running a command that can be as scary as the following (it’s an example don’t run it):
wget -q http://wine.budgetdedicated.com/apt/387EE263.gpg -O- | sudo apt-key add -
The solution
Why not extending apt-url to install a repository, and possibly the keyring file in place of the user ? You click on a web-page, you get a confirmal dialog and, if you accept it, you get the repository added and the keyfile imported.
Now ask the question, “Will we ever see this on Ubuntu ?” Well, I have three news for you:
The good:
Ubuntu 9.04 will have click-to-install repositories. You click a link, and it installs the repository for you.
The bad:
The repositories installable via single-click will have to be whitelisted. That means that only ubuntu-community approved repositories will work with the click install. That narrows a lot the scope of the install.
You can give a look to the guidelines for the repository approval here:
https://lists.ubuntu.com/archives/ubuntu-devel/2009-February/027355.html
The ugly:
Not only getting included seems quite hard, also:
Only packages where inclusion in the appropriate Ubuntu repository
is not feasible for some technical or licensing reason will be
considered.
In other words, we will get the feature, but only for Medibuntu :-/.
This is totally insane and incoherent. Really, what’s the point ? This will solve only the codecs problem. You’d be better set to create a deb containing an ssh script which adds the Medibuntu repositories and ask user confirmation before installing (explaining the content may be unlawful in some countries). That would be the same exact thing, no need for extending apt-url.
Why ?
Why the guidelines are so strict ? That’s because installing a new repository is potentially dangerous, even in case the source is trusted, because the contained packages may overwrite current system’s ones. That may lead to system instability.
Critique
The problem is already there. Danger is everywhere, even if it’s not yet perceived. Let’s not even talk about malware, Linux is fairly secure system, but when it comes to social engineering it’s just as vulnerable as every other system (and more).
When it comes to system-bricking by bad-engineered software, it’s still as vulnerable as others in the very moment you move away from official repositories and package. A single .deb may easily break your system, as it can easily override some other system package. That’s the reality. And as you know, .deb are already easily downloadable and installable with a few clicks.
There’s really no way a repository can be more dangerous than a single .deb package, other than – maybe – give more chances to the packager to do a mistake.
Workaroundability
Even worse, a single .deb may be easily used to entirely workaround the whitelist limitation and install one or more third-party repositories. So easily that I wonder why no one has already done that yet on Ubuntu. If any of you guys is familiar with Cydia (an apt-based installer for jailbroken iPhone apps) you’ll already know how adding a repository is done on the iPhone: you install a package which in turn installs a new repository.
Security by obscurity never works
Ultimately, making things difficult for the beginner may reduce the dangers to a certain degree and may work for a certain amount of time, but it’s always doomed to fail as the interest on a certain platform increases (like is happening on Ubuntu).
Setting up an approval procedure like the one described in the RFC linked above it’s just a shortcut to barely work around the core problems and give ourselves the illusion of having solved a problem, but at the same time:
- puts an (increasing) additional human-time cost on both parts (ubuntu and third parties) and will become a real burden in the long term.
- unnecessarily raises the entry barrier
A waste of time
The rules being so strict will have no effect but encourage workarounding. So these will likely obtain the opposite effect they are meant to have. Even if peer-review is to be considered necessary:
Only packages where inclusion in the appropriate Ubuntu repository
is not feasible for some technical or licensing reason will be considered.
..this point will put out of business everyone but the Medibuntu repository and perhaps some commercial vendor. Being Canonical’s partner repository already in place, this is gonna be used only by a bunch of closed source freeware producers (Virtualbox and a few others).
Having closed software running on my box doesn’t excite me that much, but it’s just me, anyway I’m curious to know how the not-feasible-for-licensing-reasons couples couples with:
Accepted repositories will undergo process similar to the ubuntu SRU process to ensure. This includes reviews/testing in a staging area documented in bugs.
That leaves us really only with Medibuntu and with not-feasible-for-technical-reason-software (??? if anyone has a good example about software that can’t be included for technical reason in the main repository but it’s fine to have in a third-party repository please leave a comment and let me know, I can’t think any).
What’s the additional repositories fuss all about
The real reasons because people creates extra-repositories instead of getting their software in the main repositories are:
- getting software in the official repositories is hard. You don’t have upload rights. Your package will be peer-reviewed by people who are likely doing that on their free times (with a significative time-lag as result). That’s done to ensure high standards.
- it is forbidden to deploy feature enhancement on the current version of Ubuntu. You can ask for an exception of course, but it’s likely you can deploy new features only once every 6 months. That’s of course too slow for most third-party developers (one example: wine).
Sure, there’s the case where licensing issues prevent the inclusion, but while it does affect an enormous number of users it also affects an incredible small number of repositories, in percentage.
Summing up the most common use for third party repositories is to allow users run the most up-to-date software. Users perfectly know that running up-to-date software may lessen the stability. It’s an explicit trade-off between stability and features+bugfixes, and one that many users are willing to accept.
Conclusions
The new 9.04 is overlooked because will be probably worth nothing. You’ll add whatever repository by-hand anyway.
The new repository click-to-add mechanism is a waste of developing time, since it will likely only be used for Medibuntu. The repository approval set-up is a incremental waste of time for both the community and third-party developers and will be workarounded very soon.
The user is still considered an idiot and malware will spread anyway as soon as the times are mature.
Are end users idiots ?The current unified policy of requiring root permissions for installing either system critical software as well less enduser software such as additional themes, internet browsers etc, is bad(tm) and will be the entry door for malware to spread into Linux systems.
Anyone who says that Linux will never get viruses or malware because it’s secure is not to be trusted. Ever.
While we bother our users Microsoft is catching up on point and clickyness. Fast.
Dude, great post. I recently “implemented” apt urls in my blog for easy installation of packages, but I only have links for stuff in the official repositories. Of course in the process, I thought on how stupid it was that there was no easy way to install repositories.
Whitelisting the repositories isn’t really as bad a concept as you are putting it. Putting a warning saying this may harm your computer is faulty by design: users ignore it, click ok and go on with their lives. Whitelisting admits there is a real security issue. You are completely right on how restrictive the policy regarding approval is, let’s hope it changes. If you can get the PPAs in launchpad approved, stuff I’ve seen in google code, etc, that would be a very good advance.
I am missing the justification as to why the users would be considered idiots? The guidelines seem to me to be a reasonable, why wouldn’t you want a package going through an SRU process?
Sorry, but I stopped reading at ” you’ll already know how adding a repository is done on the iPhone: you install a package which in turn installs a new repository.” – which to me means that you think this is some kind of magic these Cydia people came up with for apt. They didn’t, anyone can do it right now, and in fact playdeb.net even does it (install a .deb and it adds itself to the sources).
@Vadim P.: “which to me means that you think this is some kind of magic these Cydia people came up with for apt. “ Wrong. I was trying to pointing out it’s a trick that anyone can do.
@Jorge: That really make sense in the case of a repository containing packages not includeable in Ubuntu official repositories. What I am missing myself it’s the reason why only those kind of repositories will be accepted for whitelisting.
Anyone who has ever worked in tech-support knows, yes, users are idiots. Sorry, but they are
@Kirrus:
The example you give for importing a key file is wrong, that commands adds a repository.
And I think adding a separate file to sources.list.d is much better than editing sources.lists IMO.
@Vadim P. and @Stefano, you say that adding a repository using a deb file is really simple.
If that is the case, why is this apt-url support for repositories such a big issue?
Can you send me some pointers on how is this done? I tried doing this in the past and got 90% of the way (putting the repository file into sources.list.d and also importing the key). I got stuck with reloading the package information (apt-get update), not really possible from a post-install script (at least I could not figure it out).
Playdeb, from what I can tell, is not really doing this. It firsts asks you to install a custom version of apturl, and then a separate install for the repo (but no key and no reload). So it is a major hack.
@Marius: thanks for pointing out my mistake.
I’m with you on the separate file, interestingly the GUI adds on the sources.list file, that’s why I stuck with that in the command line example.
On a separate note, I regreted having wrote such a lenghty introduction, as it only distracts from what it’s pointed out from rest on the post.
@Marius: I didn’t approach the issue myself, but when thinking about it I foresaw a possible apt-locking problem (if it’s what you’re talking about).
Sure it’s a problem, but I believe it’s overcomeable in a way or another.
From a different point of view, I think that adding a repository with a .deb is an hack itself, so I believe that’s the reason of the need of extending apt-url.
Sadly I can’t help you with that, having never tried to do it myself.
The bit on Pidgin on getdeb, and how you have to install packages in a particular order… I found that to be highly exaggerated. Or not, depending on what you try to do.
What I do when I install the latest version of Pidgin is just to put all the .debs in a folder then run dpkg -i *.deb in that folder. That seems to work perfectly. Also, if I download a .deb package that requires more packages to be installed, a dpkg -i run with that .deb as the argument will mostly result in a corrupt (unconfigured) installation, but nothing that can’t be fixed with a quick aptitude install -f, and then select a solution that resolves the dependency problem by installing more software instead of removing the one that was just installed (usually the second solution presented).
Piece o’ cake!
@Victor: maybe I should rewrite my writing style then
.
By the way, what I meant in the introduction, is that .deb is unpratical for distributing packages with dependancies not present in the official repositories.
Also, keep in mind we’re talking about end users, that may not know shell commands like dpkg. Not that command line is bad, but it’s outside the domain we’re discussing about.
Should an end user mess with command line just to get the latest and greatest version of a software ? Sure that may be less stable, but the opposite may be true, and important bugfixes may be included.
Your opinion has been greatly appreciated !