My requirements for an "ideal" file sharing network

The following is a repost that I made on Reddit's programming sub-reddit. The original discussion is here.

-----------

I feel like there are a bunch of ideas floating around right now and if they were combined in the right way, then the game is over.

Any successful sustainable file sharing needs to offer the following features:

• High performance (this is why Kazaa, Limewire, Bittorrent, and Rapidshare all did so well in their respective heyday periods, slow will always fail like Freenet and Tor's current state)
• Anonymity between the uploaders and the downloaders to prevent MAFIAA B.S.
• Useful search tools (any number of solutions for this, but it is tricky on a DHT system)
• Poisoning resistance (basically, better hashing than was done in Kazaa, modern stuff generally has this solved)
• Incentives to share (the BT trackers get this right)
• All traffic encrypted to prevent ISPs and proxies from observing traffic (Tor/Freenet style to keep everyone that forwards traffic blind)
• Prefer to keep everything internal to the network and have no concept of exit nodes (like Tor's hidden services or something similar)
• Open to anyone that wants to join without invites (this eliminates friend-to-friend/darknets)
• Flexible routing and robust storage (DHTs are great for this, also makes F2F useless IMHO)
• Completely open source and decentralized so no companies can be sued to take the network down (BT already figured this out). More on this here: http://www.eff.org/files/p2p_copyright_wp_v5_0.pdf

There may be more requirements, but I feel like if all of these were combined, then the game is over and the nerds win. In particular, with anonymous communication and incentives to upload, downloaders can just leave all of their files available permanently, vastly increasing the performance, which further adds to the utility in a positive feedback loop.

Obviously, some of these requirements seem to conflict at first glance. But given how fast P2P systems have come in a decade, which includes the entire history from Napster to DHTs to everything else, I think they will be overcome in the relatively near future. This is a classic example of how arms races promote technology. I feel that without extremely draconian laws that outlaw cryptography and network neutrality, there is absolutely no way for big content to win this in the long run. Digital information is, for all practice purposes, a public good (it is non-rival and non-exclusive thanks to the ability to make infinite copies for almost free). The legal system is set up assuming that IP is a private good and will not change unless it is forced by outside pressure. Eventually, they will either accept file sharing or not. But if the technology makes enforcement impossible, it really doesn't matter.

One protocol that looks promising to me is BitBlinder. I have not been able to test it yet though (haven't got an invite), but the incentive structure they present might be enough to get a fast onion routing network working. Although onion routing adds computation overhead and 3x or more latency, in theory it should be able to sustain high throughput (which is all we care about for high performance file sharing) if there are enough routers in the network.

Please reply with comments on the issues I mentioned, any additional requirements you can think of, any systems you know of that meet these requirements, and anything else related.

Note: I am not here to discuss the morality of file sharing. I am not even a file sharer myself (haven't even used BT in a couple years). I just want to see innovation and see some good old fashioned creative destruction. In Reddit's meme language: I just want to see the world burn.

First Post

Let's get this out of the way. I'm not starting a blog to blab about my life (who cares what I eat or how drunk I got on a Wednesday night? Damn you Twitter and Facebook). I am here to do commentary and analysis about society. Yes, there are plenty of other people doing the same thing. However, I feel that I've read widely enough that I might actually have something useful to say that hasn't been said elsewhere. I'll let you be the judge.

Maybe I should be more precise. I am not introducing new ideas. I believe that even the smartest people in the world only have one or two original ideas in their lives (Einstein only had about five and he is regarded as one of the greatest geniuses of all time) and that most people have zero original thoughts. That's okay though. We are meme machines. We copy and mutate ideas to evolve them. I am here to collect ideas that I feel are useful but have not been, but should be, widely discussed. Sometimes I will also repeat things that have been widely discussed to add something (or just tear them to pieces).

A bit about me... I am in my mid-20s and am a grad student. My background is in computer science and engineering, but I am a lifelong nerd that has read widely in all kinds of fields. Like most nerds, I am addicted to websites like Wikipedia, Ars Technica, and Reddit. I became politicized a few years ago after realizing that most of the world's problems are really political rather than technical. I have been heavily influenced by the crisis of my time: the oil price explosion and economic collapse of 2008. I am a weird mix of New Deal liberalism and libertarianism. I am also an atheist with a materialistic and empirical worldview (and a dash of existentialism, it's complicated). I am particularly interested in the topics in the blog's subtitle. Anyways, that's it for my background. If a future post requires that I spill more details, I will do so at the time.

Here is a list of things that I plan on discussing:

• Serial-hybrid electric cars
• Trickle-down economics and boats
• Spectrum whitespace
• The scary "unfunded liability" of Medicare
• My criticism of the Austrian School of economics and Anarcho-Capitalism
• "Mind reading" technology
• The downfall and last grasp for air of movement conservatism
• Artificial scarcity
• Funding of public goods
• The many-core problem

Basically, most of what I want to cover is about when technology and politics collide. Overall, I am trying to increase understanding on certain issues. I will always try to be objective about the facts, but I will be very biased towards what I think is right. I might also change my mind from time to time as I learn more. I strongly encourage commenting and discussion, but keep it civil. I want this to be as high-brow as possible. Disagree without being disagreeable. If you can't deal with that, there are plenty of other places on the Internet to go to.

Thanks for reading and I hope to get some real posts up soon.