The Editor travels to Rhode Island for this year’s Intelec.
With the fifth anniversary of the destruction of the twin towers of the World Trade Center falling on the opening day of Intelec 06, it was going to be hard to avoid the subject; so why not build it into the programme?
By now you’d think that everything that could be written or said about 9/11 had been, but the anniversary inspired the conference organisers to set up a workshop on disaster recovery.
A story most of the public don’t know is that in the aftermath of the collapse of the towers, one of the major telecommunications centres in New York – 140 West Street, not even a block away – was devastated by the World Trade Center collapse, putting 14,000 business customers and 20,000 domestic customers out of service. One of those customers was Wall Street.
This may seem trivial in comparison with the loss of life on the day, but as Charles Romano, director of technical support, network power for Verizon – the operator of 140 West Street – explained, President Bush himself called the CEO of the company and wanted to know if Wall Street could be operating again within a week. Without telecommunications that would be impossible; without power even less so. It fell to Romano and a huge team of technicians to achieve the impossible.
Photographs from the scene could only convey a fraction of the chaos that faced the engineering team. The building had been punctured by huge steel beams crashing from the towers; its basements (which contained standby generating plant) flooded; a 2.5MW generator on the 20th floor had been choked and put out of commission by concrete dust. There was no external power but, amazingly, the digital switches which ran the operation had survived. If power could be restored from the outside and the rectifier’s battery strings brought back on line, then service could be restored.
“The key to it all was having people around who could identify the AC circuits so that electricians could connect an array of diesel gensets, brought in within hours,” says Romano. By day three after the disaster, Verizon had placed 3.3MW of generating capacity on the street to power up the building, and Wall Street was operating a week later.
What lessons have been learned? You sometimes need huge amounts of gensets in a hurry. Verizon has now built eight mobile generator vehicles each with 800kWh of generating capacity. They are tagged using the GPS system and can brought to any part of the USA within 24 hours.
And that you are very dependent on workers’ intimate knowledge of the systems. In a business which builds its systems in a robust manner to begin with, ‘disaster recovery’ is merely a euphemism for large-scale power blackouts.
Telecom companies know how to deal with those, though Romano had to admit that this event, in which such total system devastation had taken place, was in a new order of magnitude – a cataclysmic event. Total cost of restoration: in excess of $2 billion.
But with the devastation of Hurricane Katrina, perhaps the cataclysmic effect is not so unusual. What became clear in this somewhat mawkish start to the annual telecom power meeting is that such events can’t be planned for and, when it comes down to it, recovery is very much based on goodwill and acts of human generosity, from suppliers, customers and contractors. If any good has come out of it, some senior executives are much more aware of the importance of standby power in their infrastructure and, as one delegate put it, “It helps shake the money tree.”
But Verizon for one hasn’t put aside any budget for future cataclysmic effects. With the prospect one day of a massive ‘quake in California, more hurricanes and who knows what from the Department of Scaremongering and Security, you’d think that somebody was really planning for a rainy day. Sometimes at these meetings you wonder if anyone is planning anything more than the next Intelec!
Workshops, free and frank discussion and the exchange of ideas are what used to make Intelec buzz. But the world has changed a lot since 1977 when Intelec began as a conference series (or so I’m told, because I wasn’t there then). I’ve only attended workshops and Intelecs for the past ten years.
Take this year’s workshop on future energy requirements for telecom systems, dutifully organised by George Zguris of Hollingsworth and Vose and Rod Shane of East Penn. What I – and I believe a lot of other people in the room, especially suppliers – were hoping to learn was what the telecom operators were likely to provide in terms of services, the equipment to provide it, and of course the plant they’d need to power it.
Well, we must have been unbelievably naïve. Like Sprint is going to role out its technology and marketing roadmap to Verizon and vice versa? Gentlemen, gentlemen, it wouldn’t actually do any harm because we all have a very good idea of the kinds of technologies you will be both deploying, so if you shared that information with all the vendors, it might actually result in some reduced costs!
Of course all this is too simple. What we had was the usual ‘dialogue of the deaf’ with brief presentations from a number of players including Brad Roberts of S & C, Jamie McDougal of Bell Canada and Subhas Chalasani, the lithium battery specialist whose details my database cannot keep track of. What they all had to say was interesting in parts, except that it didn’t address the topic.
The key points as I saw them had already been brought up by a number of presenters.
“To back up or not to back up?” is a serious question. In a session which would follow the next day, Curtis Ashton of Qwest communications was to follow up his Battcon paper (see BEST, Summer 2006 page 87) with a well-constructed argument that questioned the value of putting DC back-up on some of the ‘triple play’ (video, broadband and phone in one package) equipment located close to customers. He contended that when the power to the equipment failed the chances were that power in the home failed too, and very few domestic customers used UPS systems.
Ashton argued that if any kind of back-up was used, simply to deal with the small power glitches that caused cable modems to re-train, it could easily be effected using supercapacitors – just a couple of seconds bridging is all that’s needed.
But without central office back-up, how do you keep emergency services running – and for how long – without big batteries? Clearly a regulatory argument, but not one any of the operators present in the assembled throng cared to comment on.
We had the usual cries for more monitoring, including the old chestnut (from a man from Tyco) that lead-acid batteries would always be dumb, even though it’s common knowledge that one lead-acid maker is working with a well-known telecom equipment maker to make its dumb batteries smart!
I wonder what this remark had to do with anything. You might have been at an old style Battcon – lots of banter, but no real conclusions.
You don’t have to be all that smart to see what the answer is going to be. At least you’re following the progress of telecom providers from moving from a plain old telephone service to all-singing all-dancing broadband, HD TV and voice-over-IP.
There was an answer, and it came in one of the most poorly attended battery sessions of the meeting – if there were 20 people in the room, I’d be surprised. Perhaps it was the paper title ‘Enclosure space reclamation ABCs’, from Emerson Network Power, delivered by Wesley Bydeweg.
The take-home message came with a tremendous and witty preamble, but it’s basically this: the distributed nature of digital telecom services has run into something of a space problem. There’s virtually no more room in the equipment racks to put in what the companies want – revenue earning broadband circuits. Instead the cabinets are full of ‘bloody batteries’.
So the solution is obvious – put the batteries outside. And that, of course, tells you everything. You are not going to get temperature-sensitive valve-regulated batteries lasting very long if you put them in a box, outside of your main cabinet, without any environmental control.
So what Emerson Network Power has done is to put a bank of Avestor LMP batteries in a box outside of the main cabinet. They’ll have no problem dealing with the extremes of heat and cold likely to be found in Canada, and they’re smart too. Battery consultant Katherine Bullock wondered if it was really possible to get away without any environmental controls, especially when you consider the humidity issue.
But Emerson hasn’t had any problems. In fact there have been a few benefits. Once you take the lead-acid batteries out of the equipment racks, it turns out you can run the building with a little less air conditioning – a fact pointed out by Jim McDowall at Intelec last year. It was a means of justifying the high up-front cost of SAFT’s Intensium lithium-ion system with a cellular phone operator.
That argument has got even stronger for SAFT this year, given the rise in energy costs such that the company claims you can get a payback with a switch to lithium for back-up, and much less air conditioning, in a year! Isn’t that what Americans call a ‘no brainer’? It just depends on how big companies are structured. If the people who are trying to cut energy costs get to talk with those who are trying to change energy storage policy, and they both get to speak to the bean counters, things might happen very quickly.
Bydeweg thought that the future didn’t look particularly bright for lead-acid in telecom back-up, and when I looked skywards later to speak with Jim, he believed lithium-ion for telecom had now reached a tipping point.
It was a very important paper that deserved a much bigger audience. The end is far from over for lead-acid in telecom standby, but this could really be a beginning for the lithium technologies.
Generally the battery papers were dreadful, because there was so little new to say – especially in the lead-acid session – but, surprisingly, the fuel cell sessions were informative for all the wrong reasons. Mike Perry of UTC, one of the oldest names in fuel cell manufacture in the USA, did a quick straw poll among his audience: how many outside of those working for fuel cell companies knew of the variety of types of fuel cell? Answer: very few.
The fledgling industry still has a mountain to climb if it believes it can make a business case here. The technology is not a substitute for batteries, but potentially a replacement for generators. Its quasi ‘green credentials’ are being exploited by companies that want to look good in the eyes of various stakeholders – just don’t ask too many questions on where the hydrogen came from, or the well-to-wheel efficiency of the whole process.
Pretty much every presentation here was an education job – well ‘education’ is how the vendors saw it. Fuel cells potentially combine the best features of engine-driven generators and batteries (yes, I know, on everything except price and complexity).
Perry’s presentation steered the audience towards PEM cells and hydrogen fuel (because that’s what the company does) and yes, the company has simplified the technology as much as it can (it’s been in development for over ten years).
But it is cost and reliability that are going to be key, and Perry’s paper provided a bit of a po-faced answer. Are PEMFC-based back-up power systems cost effective? It depends on the requirements of the specific application. For example, if relatively long run times are required (say, around eight hours) and using a generator is considered unacceptable, then a PEMFC solution may be attractive, especially where the average power requirement is relatively small (e.g. ≤ 20kW). Mid-sized energy storage (e.g. in the order of 10 kWh) is ideal for a fuel cell based system.
The main reason for this is that the cost of adding additional hydrogen storage is cheaper than adding more batteries. This is why telecom applications look especially attractive for a PEMFC, since there are many instances where highly reliable back-up power is required and the average power output is relatively small, but the runtime requirements are relatively long.
In quantitative terms, a PEMFC back-up power product that costs US$2,000/kW can be cost competitive for some telecom applications, on a lifecycle basis, with a battery-based solution. But here comes the hard facts. No fuel cell company can actually produce a complete PEMFC system at this cost today, because of the low volumes of units currently being sold.
Fortunately the US Energy Policy Act of 2005 includes a federal tax credit of $1,000/kW (or up to 30% of the installed cost, whichever is less). So, for end users who can take advantage of this US federal tax credit, the cost of a 5kW system priced at US$15,000 (plus US$1,500 for installation) is actually around US$2,300 per installed kW.
So according to Perry, in the near term, a natural price point for fuel-cell manufacturers who are still struggling to get their costs down is US$3,000/kW, in order to take full advantage of the federal tax credit. In the longer term exceeding these initial cost targets should be no problem since, claims Perry, there is nothing inherent in a PEMFC system that makes meeting these costs difficult.
Keep in mind that the cost target for automotive applications is US$30/kW and automotive companies are investing billions of dollars to develop PEMFC-powered vehicles because they view this orders-of-magnitude lower cost target as achievable in automotive volumes. In fact, a US DOE-funded study projects that the cost of a current state of-the-art automotive PEMFC system would cost US$110/kW, based on a hypothetical production volume of 500,000 units per year, and in 2002 the cost would have been $275/kW at the same annual volume.
Clearly, rapid progress is being made in PEMFC technology; but a lot of fuel cells are going to have to be sold. BEST wonders where they are all going to be used. It’s all rather chicken-and-egg.
But technically fuel cells are rapidly becoming the last technical frontiers of interest in this 30-year-old conference series, whether they have any commercial merit for telecom operators or not.
If a standby power meeting can’t get more relevant technical battery papers than this one did, you have to wonder what it’s purpose is. I hope the organisers in Rome can provide tastier fare.
