Sips from the Firehose
A blog that seeks to filter the internet into a refreshing, easily-gulped beverage

Nov 29

4.8 Zettabytes of Content -or- Why We’re Really REALLY Gonna Need Good Filters

Posted: under Digital Migration, Science, Vastness.

One of my oft-repeated memes for my trainees – and, not coincidentally, the reason behind the very name of this blog – is that as we move further into these uncharted digital waters, what we need more than more information is better filters for the ever-increasing torrent being directed at our frontal lobes.

So imagine my weary non-surprise at the latest mind-boggling numbers about the sheer amount of data/content/1s and 0s streaming through all these bundles of fiber we’ve laid around the world.

Way, way back in the early personal computing days, I remember having a conversation with a fellow geek. He was talking about the latest advances in floppy disk technology; and yes, these were still the old 5 1/4″ floppy disks that actually, well … flopped around when you waggled them in your hand. Not that you wanted to do that very much – the thin plastic disk inside was prone to slip around and maybe even crinkle a bit.

Anyway – we were talking about how the amount of data you could store on such a disk was about to increase from the then-standard 360K disks were almost due to an upgrade, so’s they could store 720K.


And then … some day in the near future (his eyes grew distant, focusing on such a massive, magical shift in storage technology) … “You know what comes after that?”

His voice grew hushed. Almost reverent.

“A megabyte!”

At the time, I was writing programs in BASIC to be executed in the 16K space available on our old TRS-80s. I was impressed. With a megabyte of storage … wow. The possibilities were endless.

Skip to this news. With the sheer amount of content streaming down all these here Intertubes:

In terms of pure data center traffic around the world, traffic is
projected to go from 1.1 ZBs in 2010 to an estimated 4.8 ZBs in 2015,
four times the amount. 4.8 ZBs is a hard-to-imagine number, so Cisco has
quantified it to equal 66.7 trillion hours of streaming music at 160
kbps, 15.5 trillion hours of standard-def web conferencing or 4.8
trillion hours of online streaming 720p HD video.

…the need is going to be ever-greater for The People Formerly Known As Journalists to filter that torrent. To use their human judgment to identify what is important and worth paying attention to from that which is not.

Or, as Clay Shirky, so eloquently puts it:

Web 2.0 Expo NY: Clay Shirky ( It’s Not Information Overload. It’s Filter Failure.

, , , , , , ,

Comments (0)

Aug 10

FCC Getting Annoyed About Slow Broadband Roll-out: Is the Answer a Purple Laser?

Posted: under New Media Strategery, Science, Web/Tech, Webconomics.
Tags: , , , ,

$8 billion a year to POTS; “we are no longer on the right track”

Anyone who’s traveled around the world has probably noticed what Janine and I have these last couple of years: we can usually access the internet much faster in other countries than we can here in the good ol’ US of A, where the internet was invented (take a bow, Al Gore!). When we were in Costa Rica, even in a hotel lobby, web pages just zoomed into view; we attributed the speed to the massive online gambling infrastructure that’s been built in Costa Rica recently. (It’ll be interesting to see what happens long-term to Costa Rica; it’s my hope that the law of unintended consquences will kick in, and the somewhat sordid gambling biz will actually result in more legit businesses using that bandwidth to grow & flourish.)

Anyway, a report this week from the FCC is, in the suble-bordering-on-inscrutable language known as “Bureaucratese” a cattle prod to the backsides of all the various carriers, cablers, telcos and gougers currently charging fat fees for puny bandwidth. Herein a sample:

“The report points out the great broadband successes in the United States, including as many as 290 million Americans who have gained access to broadband over the past decade,” FCC Chairman Julius Genachowski said. “But the
statute requires more. It requires the agency to reach a conclusion about whether all — not some, not most — Americans are being served in a reasonable and timely fashion.”

That’s not happening, he added.

Additionally, it appears that the revenues from the tax on long-distance service that we all grit our teeth and pay each month, and that was supposedly earmarked for improving service just like this has been instead diverted to Plain Old Telephone service (POTS).

More and more, we’re seeing governmental agencies starting to recognize that bringing high-speed internet to communities is an essential ingredient to lifting the local economy. This might have particular impact in the rural areas of the U.S. where coverage is lagging (and where the challenges are most severe), because the farmers/loggers/fishermen might be able to circumvent the supply bottlenecks that are eating up any hope of profits.

Still, I am reminded of the statistic that was widely quoted early last decade, where AT&T got the “gulp-adjust-your-collar” number of $90 billion just for the landscaping costs of stringing fiber-optic in the Western U.S. So what’s the solution?

Well, an interesting experiment was featured on Scobelizer – and the genesis was the big skyward-pointing light atop the Luxor Casino in Las Vegas. As I understand it, a giant laser system in the purple band could provide more than five (5) times the bandwidth than even the fiber-optic lines (Fiber To The House or “FTTH”) that are the fervent dream of all us techno-nerds still being held captive by Time-Warner Cable/Adelphia/Comcast/whatever. Basically, the information is streamed up into the sky, and

A purple laser which is almost invisible to the human eye and which is
inexpensive to buy (they are the lasers inside every Blu-Ray disk player
— the lasers are actually purple light, the “blu” in the name is
marketing) is aimed at the sky and an array of sensors reads data from
the beam of light. Readable due to scattering of light due to the
atmosphere. He showed me how this works: you aim a laser at the sky and
everyone can see the beam. If your human eye can see it, sensors can see
it too and due to some tricks can get massive amounts of bandwidth out
of the laser.

What would this mean for mobile bandwidth? Plenty. The problems I’ve seen with cell coverage in rural areas have less to do with the bandwidth coming from the towers than they do with the capabilities of the radios in the handsets to make the connections. Or, to put it another way, if you make the transmitter in your mobile strong enough to send a signal to a tower 4 miles away, it’s also strong enough to make the hair on the side of your head warm from the microwaves (anyone else remember this phenomenon?). Or to cook your retinas.

But if the bandwidth/connectivity issues can be solved by having some cheap Wi-Fi routers spaced around, connected to sensors pointing at the purple laser beam, then all of a sudden, we have a lot faster, cheaper and more reliable coverage. Even having a little Blu-ray laser integrated into the various existing 3G antenna arrays would be a massive improvement (if their various whitepapers aren’t just hokum).

This could really have an effect in some of the more rugged countries that I’ve done work in – I’m thinking of the mountainous regions of Chile, Colombia, Kazakhstan, and most recently, Georgia. The upstream bandwidth is probably still pretty limited, so in a certain sense, this is just a variation on the DirecTV/satellite internet service paradigm, but still, most users tend to download about 1,000 times more information than they upload.

Comments (0)

Dec 07

Spam 2.0: The Bad Guys Get Trickier

Posted: under Online (Multi)Media, Science, Web/Tech, Weblogs.

I was wondering WTF was up with all the goddam spam lately. I had chalked it up as just being symptomatic of the Christmas season, when retailers, online and off, all hawk their wares. Apparently, not.

The big problem these days is that the spammers have enlisted an army of robot zombie computers to do their evil bidding.  Which, in a Beavis/Butthead kind of way is a cool concept. If only they were doing it to accomplish world domination or get rid of sucky hair bands, then that’s be OK.

Unfortunately, the spam these days is all in the “Hi, it’s Frieda” subject line, with an image of the text about some shitty pump’n’dump stock or cheap-ass imitation Viagra.

The use of botnets to send spam would not matter as much if e-mail
filters could still make effective use of the second spam-fighting
strategy: analyzing the content of an incoming message. Traditional
antispam software examines the words in a text message and, using
statistical techniques, determines if the words are more likely to make
up a legitimate message or a piece of spam.

The explosion of
image spam this year has largely thwarted that approach. Spammers have
used images in their messages for years, in most cases to offer a peek
at a pornographic Web site, or to illustrate the effectiveness of their
miracle drugs. But as more of their text-based messages started being
blocked, spammers searched for new methods and realized that putting
their words inside the image could frustrate text filtering. The use of
other people’s computers to send their bandwidth-hogging e-mail made
the tactic practical.

It is amazing the techiques that spammers have adopted – apparently, the software that stopped spam relied on finding the same words in the same messages – that is, if you send out a million messages all saying “Eat at Joe’s,” the servers would recognize that message and kill your spam.  The spammers got around that by programming code that would take random snippets of text and stick that in the message beneath their sleazy come-on. 

Now that it’s images that are being sent, the bots change AS LITTLE AS ONE PIXEL in a picture, or put in noise and scratches so the OCR software can’t zero in on the craptastic ad. 

Why do the spammers continue to bother?  Well, it pays off.  Sadly, the average netuser is still an utter and complete dunderhead.

Though the scam sounds obvious, a joint study by researchers at Purdue University and Oxford University
this summer found that spam stock cons work. Enough recipients buy the
stock that spammers can make a 5 percent to 6 percent return in two
days, the study concluded.

This is really depressing. The fact that there are still people out there dumb enough to click on an attached image in a mail message obviously not meant for them, and to go “Hmm.  Wowee, Myrtle, lookee here, there’s a stock that is just about to go ballistic. We gotta get in on the ground floor o’ that!”

The adage about suckers should be updated to “A sucker and his money were lucky to get together in the first place.”

Technorati Tags: , , , , , , ,

powered by performancing firefox

Comments (0)

Oct 17

I’m An Eloi Before My Time, Apparently

Posted: under Current Affairs, Online (Multi)Media, Science, Weblogs.

This was a study for Bravo about the future of evolution.  Apparently, in the
future, there are going to be Eloi, who will look … pretty much like me. 
Lucky them.Meanwhile, the rest of the folks are going to look like … uh … Dick

There is a distinctive whiff of "Brave New World" about this – well, that and "The Marching Morons."  And of course, "Idiocracy."

I think that this guy has some unresolved issues dating back to the stack of Gent magazines and Fantasy & Science Fiction that his mom discovered under his bed when he was 14, threw out, and forbade him ever to read again.  I mean, it’s way way simplistic to say that the smart, attractive people will only breed with the other smart, attractive people … hey, anyone that’s ever wandered through the pool area around the Beverly Hills Hotel on a Sunday afternoon can see that that is definitely not true … viz all the "troll" dudes with the tall Barbie/super-shiksas on their arms … still, the study has done the one thing that it was designed to do – and that’s to call a lot of free attention to the Bravo episode.  Smart marketing, guys.

Now can we figure out how to get the posters on the Yahoo message boards stuffed back down their Morlock holes?

>>The descendants of the genetic upper class would be tall, slim,
healthy, attractive, intelligent, and creative and a far cry from the
"underclass" humans who would have evolved into dim-witted, ugly, squat
goblin-like creatures. But in the nearer future, humans will evolve in
1,000 years into giants between 6ft and 7ft tall, he predicts, while
life-spans will have extended to 120 years, Dr Curry claims.

appearance, driven by indicators of health, youth and fertility, will
improve, he says, while men will exhibit symmetrical facial features,
look athletic, and have squarer jaws, deeper voices and bigger penises.

on the other hand, will develop lighter, smooth, hairless skin, large
clear eyes, pert breasts, glossy hair, and even features, he adds.<<

powered by performancing firefox

Comments (0)

Mar 09

Spy on Yourself From the Comfort of Your Home

Posted: under Current Affairs, journalism, Online (Multi)Media, Politix, Science, Web/Tech.

Surveillance …and no, this is not just another rambling screed about how the NSA domestic spying program is TEOTWAWKI. Although that’s always fun.

No, this is about the fumbling attempts to try to figure out how to deliver the news/information/entertainment (and they are all one now, let’s just accept it and move on) that you want and need. As has been endlessly noted, by derivative hacks like myself, the information flow on the ‘Net is all about breaking up the hieirarchy of "Me talk – You listen" that the old media was all about.  Yay for us.

Now then.

The problem becomes how do the various news/information/entertainment (hereinafter referred to as "NIE") providers get your what you want?  And how do you refrain from stumbling and bumbling all over the landscape while real cool stuff that you would love to know/see/hear lie undiscovered at your elbow?  How do we make this New Media thing work in a way that isn’t about shoveling crap at you or having to wade through garbage to get to the jewels?

One solution is for some outside agency or company to track what it is that you look at and then try to give you more. 

I like this idea so much that I’ve yanked the phone cord out of the back of my TiVo so it can’t rat me out to the Corporate Overlords and clog up the hard drive with "suggestions".

I put up with the list of "hey, you liked this book – you might like these other ones that probably aren’t as good, but have neat cover illustrations" because I don’t buy all that much stuff from Amazon and I figure that if I spread out my purchases they can’t get a real good bead on me.

But still.  Makes me a bit uncomfortable.

This might be the answer – and the full confession is that I kyped this off BoingBoing

Musical Myware, Felix Miller, CEO Felix introduced me to the term "myware" — spyware that you run on your own activities, which helps you get a better handle on your needs and wants and helps your computer help you better.

I love this idea: people are good at making decisions and computers are good at counting them (and computers are bad at making decisions and people are bad at counting them). My computer should note, count and process every decision I make — it should notice that I never answer emials from certain people, it should notice that I never click through certain stories in my reader, that I load certain pages every day, that I often search my mailbox for certain kinds of messages and so on. That’s stuff I’m totally unqualified to keep track of, and that computers are really good at:

Myware is like spyware, but it lets you spy on yourself.

Why would you spy on yourself? Why would you share the data with Tell us what music you listen to, anytime, all the time, without even realizing it


Napster made all music ever recorded available — so how do you know what to listen to? Mission: "Harness the knowledge of the crowd." Someone out there knows what you should be listening to; no need to read tedious editorial.

Audioscrobbler installs in media-players like iTunes, etc and reports what you’re listening to at any moment and updates your user-profile. Only records stuff you listen to, but not stuff you skip — just the stuff you pay attention to.

This has real promise.  If there’s something in my local computer box that tracks what I look at and find interesting, and then sends its tendrils out to find other stuff like that … but only when I want it to, and only in ways that I find acceptable … and I can actually make a few coin by licensing what I want/like/need … well, that’s OK then. Until some script kiddies start screwing around and slurping all that data off my hard drive with a bot.

There’s a good space here for someone coming up with something using the meme "blind date" where your computer "fixes you up" with content that they think that you’ll like. ("Really, this story has a great personality! Yeah, so maybe it could use a little slimming down in a few places, but it’s really sweet! Trust me!")

More on this as I recover from last night’s multi-birthday party.

Comments (2)

Feb 24

Blink, Hypnotism, Hidden Persuaders and Your Future As a Robot

Posted: under Current Affairs, journalism, Online (Multi)Media, Politix, Science, television, Web/Tech.

OK, most of the time I’m pretty strongly skeptical of conspiracy-theory nuttiness.  Working in close proximity to actual Men in Black, I have observed that most of them are lucky if they can all agree on where to get the morning donuts.

This is not to say that Deep, Dark & Deadly things do not happen.  But the plotting for them is mostly spur of the moment.  And they stay out of the public eye not because of some conspiracy to kidnap, intimidate and silence the press, but because most reporters are pretty overworked and don’t really have the time to spend on some ‘investigative reporter chasing the Hulk’ obsession.

*[As an aside, I have to wonder if there ever was a time wherein a reporter could, like Jack McGee in the cheesy old 70s TV drama, devote every waking hour to chasing down some pet project. Back in the days when newspapers had huge staffs, did this actually happen?  I mean, back when I arrived in L.A. back in ’89, reporters muttered about staffers in the L.A. Times that produced maybe three stories a year… this quite obviously DOES NOT HAPPEN ANYMORE. OK, pointless aside over.]*

Anyway, Malcolm Gladwell has now joined the blogosphere, where he will no doubt quickly find that the signal/noise ratio is somewhat higher than it is at the New Yorker.  But reading through the first couple of postings on his blog led me down some strange thought patterns and hyperlinks – all kicked off by what he wrote in Blink and the way that I have seen those lessons either applied or ignored.

As I wrote on Gladwell’s board, there is a significant story that keeps surfacing now&again about how researchers are using CAT/MRI scans to study the electrochemical changes in our brains that occur when we are going through our decision-making tree.  Basically, they are mapping out what stimuli make us do what – the hidden neurological/biological hardwiring that is common to all of us.

It’s as if there is a secret combination in our heads – if this were a late 60s sci-fi movie influenced by too many acid trips, the evil genius would be blathering on about how colors, musical tones, smells and other sensory input directed at a person in the right combination can turn us from logical thinking beings into doing Whatever They Say (ooo-eee-ooo).

Of course, there are some things that we have discovered over centuries, nay millennia, of trial-and-error experimentation.  Such as religious experiences.  I went to a Buddhist prayer meeting years ago in a big hall filled with about 5,000 people on Sunset Blvd.  I later learned that what they do is something that can be deconstructed – but at the time, it made me feel like I was having a transcendant experience.

Apparently, if you walk out of a brightly lit room through a narrow passageway andIraqi_underground_temple come out into a large, cavernous space – where there’s a big stage that’s the only illuminated area – and there is steady, deep, rhythmic, thrumming tones (like a big gong) … this whole process relaxes you and puts you into some kind of pre-hypnotic state.  Maybe it’s like some kind of recapitulation of the birth process.  I dunno. I’ll have to look into that one a bit more and maybe post about it again later.

So today, I started thinking about that in relation to Blink – and about how that book’s message is a somewhat fractured and conflicting one. Basically, it says that we all make our decisions based on sensory input that we don’t realize we’re taking in – and that that input amounts to so much more information than our conscious brain can process.  And that those snap decisions are sometimes very very useful.  Except in those cases when our subconscious gets fooled, in which case they aren’t.

One poster pointed out this dichotomy and it got me thinking.  So I poked around a bit and read about how advertisers were (and presumably are) doing things like hypnotizing their test-market research subjects so the people won’t lie to them … and so they can better figure out what our sub/unconscious is prompting us to do … so they can sell us cereal, airline travel, Air Jordan sneakers and presidential candidates without us ever really getting to engage those pesky conscious evaluative parts of our brains.

Where I’m going with this is to pose a question: is this something to be at all alarmed about? Because when you take the kind of power over a person’s thinking process and combine it with the "behavioral/personality modeling" stuff that we’ve been hearing about (i.e. you combine a person’s phone calls, shopping habits, magazine subscriptions, web searches, zip code and car registration to build a predictive model as to what the person is like) … and it’s starting to make me a little uneasy. 

Perhaps I’m worrying over nothing – but it would seem to me that this potentially has the ability to really allow the holder of this information to be able to target specific audience blocs to convince them to perform the desired action (watch NBC on Thursday! Support our troops! Vote Republican!)…
Is there a dark side to the questions that people like Gladwell or these researchers are asking?

Comments (0)