Wednesday, November 18, 2009

Energy Policy Thoughts

I have been thinking and reading a lot about energy policy for a couple of years now. Recently, I started to write an extremely comprehensive post on the topic, but it evolved into a small novel so I have ruled against posting it in full. Instead, I'm going to do an ongoing series of posts on energy policy. This post will establish the basis of the remainder of the posts by simply stating the problems and a high level view of what I think is the right path to take. As a note, this is written from a US-centric viewpoint, but the arguments should apply to all people.

My overall conclusion is that the public discussion on the energy issue is not even talking about the right problem. Energy is the lifeblood of industrial society. The public discussion seems to be focused heavily on making our energy usage more efficient, which essentially translates to less energy consumption per person. Although we definitely should improve energy efficiency when it is easy (such as eventually switching to LED lighting and using "smart grid" technology), this is not a real energy policy.

Before going on, we must actually define the real problem. The core problem in energy policy is that fossil fuels, which are the core of our current energy production, need to be phased out. There are three very good reasons for this (ordered in terms of importance):
  1. Climate change
  2. Oil imports are bad from a political and economic standpoint
  3. Fossil fuels are non-renewable
I'm not going to waste my time dealing with climate deniers, so please go to Real Climate or NASA to see why climate change is real and why you should take it seriously. As for the oil imports that drive the US transportation system, we are giving away an enormous amount of money every year to people that don't like us very much. This is an insanely bad idea in the long run and will have the net effect of transferring wealth and power out of the United States. The third reason is commonly referred to as peak oil. Peak oil has a lot of people worried, but I don't view it as an overly serious problem in the short term (although it absolutely is a problem in the medium and long terms).

So, given the problems, the solutions offered by the main political parties in the United States are either to deny the problem exists (the Republicans, the libertarians, and some conservative Democrats) or attempt to address the problem by putting a price on carbon emissions, switching to renewable energy such as wind and solar, and increasing energy efficiency (most of the Democrats, the Greens, etc.). Obviously, only one side even takes the issue seriously, so I am going to focus only on the mainstream liberal solution that is offered and ignore the ridiculous and insane non-solution the Republicans offer.

My big issue with the mainstream liberal approach is that it is not enough. I am not going to cite hard numbers in this post (I may dig up the data for a future post), but I will argue from a qualitative point of view why renewable energy sources and decreased per-capita energy usage will not solve the problem. The third leg of the solution, putting a price on carbon, is absolutely necessary though. The costs of carbon emissions are currently externalized, and putting a price on these emissions fixes that problem and reduces the incentive to externalize these costs.

Renewable energy typically refers to solar and wind. There are many other possible sources (geothermal, tidal, etc.), but they are not as easy to do as wind and solar with current technology. I personally like both wind and solar a lot. Once built, they provide us with what is essentially "free" energy because we do not have to provide fuel to run the generators. The fuel comes from the wind and the sun. Unfortunately, that fuel is not always there. Sometimes it is dark and calm. This is why solar and wind power are referred to as intermittent power sources. Therefore, wind and solar cannot be used to provide the base power for our electric gird because they cannot be guaranteed to be working at a given level all the time.

Unfortunately, energy in the electric grid cannot be easily stored. It has to be used shortly after it is generated or the energy is simply lost as dissipated heat. One technology that would solve this problem is grid energy storage, which is essentially large capacity, long lasting, and high current batteries. This would allow us to build an overcapacity of renewable sources to collect extra energy when the wind is blowing and the sun is shining and then store it for later. Unfortunately, grid energy story technology is not ready to go yet. Hopefully, grid energy storage will mature quickly, but we cannot be 100% certain about the time frame.

The other issue is energy efficiency. Although better insulating houses, creating a smart grid to reduce the average load, and other techniques would save lots of energy, people still need a lot of energy to live a high quality life. Making hot water, for example, takes a lot of energy because the specific heat of water is very high. This is basic physics and we can't avoid that under any circumstances. Although I am kind of waving my hands a bit on this right now (I'll grab some numbers later or you can just Google for them), my opinion is strongly leaning towards thinking that energy efficiency is not going to do enough. Additionally, new technologies keep coming out with every higher energy demands and more people on the planet are enjoying a higher standard of living. Therefore, any gains in efficiency will quickly be eaten up by new consumption. Finally, when push comes to shove, people are not willingly going to significantly reduce their standard of living, even if it means saving the planet. Humans are too short-sided and greedy by their very nature to do this.

My overall point is that the combined problems of renewables and efficiency demonstrate that the standard liberal solution to the climate crisis is not actually going to solve the energy problem. While I do think we should do both, we need more to really solve the energy problems.

My answer to this is to use a technology that has become a sort of third rail in modern American energy politics: nuclear power! I'll cover this in a lot more detail later, but I believe that if we go all out with building nuclear fission reactors to replace our coal plants in the next 10-15 years, then we can get our carbon emissions way down and not be forced to make painful cuts in our energy consumption. Of course, nuclear has its problems (complexity, upfront investment, perceived danger of meltdowns, proliferation issues, nuclear waste), but I believe that these problems are minor compared to the worst scenarios of climate change.

Traditionally, environmentalists such as the Sierra Club have been strongly against nuclear power. Although I understand their arguments, I think anyone serious about climate change has to be willing to accept the trade off that nuclear power is the least evil approach to the climate crisis that will actually solve the climate crisis. Reducing our carbon emissions is the most important goal right now. Also, most people do not realize this, but modern nuclear reactor research is significantly more advanced than the reactors currently deployed in the United States. Keep in mind that the United States has not built any new reactors in about 30 years (we stopped after Three Mile Island happened). Third generation reactors are ready to go and already deployed in Japan and fourth generation reactors are being researched and should be ready relatively soon. These newer designs significantly increase the power output of the same amount of fuel, while also significantly increasing safety and reducing nuclear waste issues (such as the Generation IV techniques that can reuse waste as fuel and transform the half-life of dangerous waste from thousands of years to tens of years). Overall, fission provides a carbon-free bridge until better solutions come along.

Finally, I think the United States needs to vastly increase its research expenditures on nuclear fusion power research. Although, even with full Manhattan/Apollo project-style spending, fusion will probably not be available commercially for 20 years, the benefits so vastly outweigh the costs that it is simply insane not to invest in this technology. Like fission, fusion does not emit any carbon. Secondly, fusion does not have the meltdown dangers or nuclear waste side effects of fission reactors. And best of all, fusion can produce vast amounts of energy from deuterium extracted from water. Wikipedia says that there is enough deuterium on Earth to provide us with energy for 150 billion years at our current consumption rates. This is longer than the universe has existed and lasts well beyond the time when the sun burns out. This also means that if we build significantly more fusion reactors than we need, we could get the marginal cost of energy down to almost zero. Basically, once we achieve mature fusion power, we can generate and waste as much energy as we want. We could even do ridiculous things like pump water out of the ocean, desalinate it, and use it to turn the Sahara desert into farmland, solving both the water and food crisis in the third world in one fell swoop. The possibilities are staggering.

Fusion does have its naysayers. The biggest argument I have seen against it is that some people think that it will never be cheap enough to compete with other technologies (in particular a highly efficient solar panel) because of the large capital costs of building the reactors. I think this argument is flawed because we don't know what the capital costs will be until we develop the technology further. This argument is like someone in 1960 saying that building an IPhone is impossible because the cost would be astronomical. Well, things change. Materials get cheaper, modeling and manufacturing gets better, and so on.

For further background on fusion, I strongly encourage you to read the following Wikipedia articles: fusion power, ITER, Polywell, and National Ignition Facility. I am very excited about the long-term prospects of fusion and hope that it happens before I am too old to enjoy it.

Well, this post also turned out to be extremely long. But trust me, it is much shorter than the original tedious post I had typed up. I hope I was able to transfer the basis of my thinking on energy policy to you. I will expand upon these ideas in future posts.

Indexing and Anonymity

This is a follow up post to my ideal file sharing network that was originally posted as a comment on Reddit. It has a bit of overlap with my previous post on this topic, but I cover some additional technical details (hopefully in an understandable way) to explain why these problems are hard.

------------

We need to completely remove any remaining weak points in file sharing. From my count, there are only two more problems to solve to make BitTorrent "perfect":

1) Decentralized indexing/searching.

This is hard because distributed hash tables (not the "Mainline DHT" tracking system, but the actual hash algorithm used to map a key to a node), which are the underlying basis of BitTorrent cannot do inexact string matching easily. This is because, like all hash algorithms, a small change in the string makes a big change in the hash key. This prevents implementing a naive decentralized Kazaa/Limewire style search (these networks use a much more primitive base protocol that allows a search to simply "flood" the network N levels deep).

There is some research being done to solve this problem (e.g. http://www.computer.org/portal/web/csdl/doi/10.1109/ICDCS.2005.44). Overall, the future looks very promising for this problem and if the BitTorrent indexing services lose future lawsuits (which they probably will), this technology will be deployed.

2) Anonymity.

This is the "golden goose" of BitTorrent. Right now, the IP addresses of file sharers are directly exposed. This leaves them vulnerable to MAFIAA lawsuits and therefore fewer people are willing to seed (and many people are scared away altogether). Fewer seeders means slower torrents. If users could be certain that there is no possibility that they can be traced to a particular upload of copyrighted material, then the number of seeders in the network should rise dramatically as users just leave their torrents on without any fears.

The best way to do anonymity in a network right now is onion routing (e.g. Tor). Unfortunately, Tor has the problem that it is very slow. This is also mainly due to legal fears and poor incentives.

A brief overview of Tor for the uninitiated... Tor works by wrapping a message up in multiple encrypted layers (the "onion") and forwarding it along to several nodes. Each node in the route decrypts the outer layer (peeling away the "onion" layers) and then forwards it to the next node in the route. The last node (a.k.a. the exit node) then handles the request and sends the result back using a "reply onion" defined by the original requester. This means that no one in the onion route can connect the requester to the server that eventually receives the request, therefore making the requester anonymous. As long as all nodes in the onion route aren't cooperating (which is hard to do in practice because the requester can select any set of nodes it wants), then this is extremely secure. However, the weakness of the system is that it provides no incentive for peers to act as internal routers and especially as exit nodes (which is dangerous from a legal perspective because much of the traffic leaving Tor is illegal). More on Tor available here.

Tor's performance is weak on both of the classic networking metrics: latency and throughput. The high latency arises from the fact that each request has to make multiple hops across the Internet before reaching the destination. There is nothing that can be done about this and therefore any application that requires low latency is not suitable for onion routing. On the throughput side, Tor does decrease throughput somewhat (because nodes are sending and receiving more data than they would otherwise), but given that connections between nodes are rarely operating at the maximum rate of transfer (and this will improve even more as fiber to the home is deployed), the throughput of a Tor-like network should be high enough to support BitTorrent-style downloads (keep in mind, latency does not matter very much in BitTorrent applications). The key issue is to ensure that the onion routing network is not maxed out because there are too many free riders that are using onion router bandwidth but not contributing.

There are attempts to address this problem. The BitBlinder system tries to create incentives using an electronic cash infrastructure to require nodes to act as onion routers. This should increase the internal bandwidth available within the network and vastly improve performance. They claim to have achieved reasonable download speeds on BitTorrent over their network (which is still being developed and is currently available as a limited release). More info here.

Another approach is to use the Friend-to-Friend darknet design. This means that each node only connects to nodes it trusts and that untrusted parties will not be able to identify the node as the source of an upload. This approach is implemented in the OneSwarm project from the University of Washington.

The key difference between the BitBlinder and OneSwarm approach is trust. In BitBlinder, no one has to be trusted, but it requires the additional overhead of full onion routing. In OneSwarm, the overhead is that one has to traverse the trust network to reach data, but given the small-world phenomenon, this should require no more than six degrees to find any Kevin Bacon movie. ;)

Note: I am not completely familiar with the OneSwarm architectural details (the papers are buried in my overly long reading list), so if someone that knows it (preferably someone that helped develop it) can fill in the details with a replay, that would be great. I'm particularly interested to know how locating and routing are performed. Do you always only talk to trusted nodes or can there ever be short circuits in the trust network? Also, can the trusted nodes observe the traffic you are sending/receiving unencrypted?

There is a third additional feature to all of this that may or may not be necessary and that is traffic obfuscation. If the legal system decides that forwarding encrypted traffic is bad and orders the ISPs to use traffic analysis to block such traffic, then we will need to disguise the P2P traffic in some way. The simplest way I can think of is to just set up a bunch of fake VoIP/Videoconferencing calls among the friend-to-friend network and then just pad the stream with fake data whenever P2P traffic is not being sent. Also, the stream can be modified to have the same traffic analysis patterns as normal VoIP/Videoconferencing traffic. There are probably other protocols that can be used as well, but I selected VoIP/Videoconferencing for this example because they send lots of data.

Conclusion: the future of P2P is very bright. Short of extremely draconian legal measures that violate net neutrality and privacy (e.g. throwing someone in jail for forwarding encrypted traffic on behalf of others with no knowledge of what the traffic is), there is no way to take out the P2P networks of the near future. The pattern is clear: every time the MAFIAA makes progress in their lawsuits, the next level of the technology is released and we are only a couple of steps away from a network in which the MAFIAA has no one to sue except for their immediate neighbors in the network that can claim with plausible deniability that they did not knowing forward any copyrighted material.