CMS Watch Search and Information Access Feed

What Wimbledon and vendor selection have in common

tregli@cmswatch.com(Theresa Regli) — Thu, 2 Jul 2009 15:26:00 -0400

As Murraymania swept the UK, I settled into my Court No. 2 seat on Wimbledon's always-action-packed middle Saturday. In addition to the matches of Serbia's Ivanovic, Australia's Hewitt, and Russia's Safina, I had a great view of the Centre Court scoreboard, so during breaks I was keenly watching the results of Andy Roddick's match.

"Andy's got the first set," I said to my cousin, who's studying in London and joined me for the day. "Andy's not playing yet," interjected the Brit to the other side of me. "Yes he is," I said. Pause. "Oh you mean, your Andy," he replied. "Right," I smiled back, "not your Andy, who plays tonight." Then came the most interesting comment: "Well, he's not really my Andy," the gent said. "I'm English, and he's Scottish."

Territorial rivalries are perhaps more pronounced in sport than any other pastime, be it the Boston Red Sox vs. the New York Yankees, the Calgary Flames vs. the Edmonton Oilers, or the New Zealand All Blacks vs. the Australian Wallabies. Such territorial rivalries aren't altogether absent from the content management vendor selection process, either, and I find this much more pronounced on the eastern side of the Atlantic than in my native North America.

Of course, in Europe and the UK there are many nations and territories (both political and historic) in a comparatively tiny geographic area, which makes a perfect petri dish for such rivalries to fester.

When I work with clients or subscribers to help them select vendors, the two or three finalists often end up being very technologically similar, and once tools have been tested and deemed appropriate for clients' environments, the conversations just prior to final selection often become very much "cultural." It's not just about whether the team is qualified, and if the support line is open when their time zone is open for business. It's also about who they are, and when a few hundred thousand sterling or euros are on the table, the rivalries come out in closed-door conversations.

"They're Belgian -- there's so many jokes about Belgium. Isn't there a reason for that?"
"They're Dutch -- so they're blunt, that's good, right? Aren't the Dutch cheap, too?"
"They're German -- so regimented -- is that right for us? What if the schedule slips, will they charge us double?"
I wouldn't even know where to start on the Scandinavian rivalries, which go back to the days when Sweden and Denmark traded off conquering the whole of Northern Europe.

I end up spending quite a bit of time talking with clients about how they can benefit from vendor characteristics that are different from how their company normally functions. A bit of German organization and Dutch bluntness can be a great thing if your company has neither. I also watch vendors make an extra effort to bring in the "local flavor" to meetings -- someone from the local country or territory, if headquarters is on the other side of the continent. This always makes a big difference to buyers -- more than I believe it should. The English sales guy in the meeting in London isn't going to be the one you'll be working with, or providing you the ongoing service you'll need. Good service is good service, regardless of where it's provided from.

As an American who does a lot of work in Europe and the UK, I also experience trepidation on the part of some buyers. "Oh, you're American," I sometimes hear when I connect with a potential client via phone or meet up in person. Well yes, but CMS Watch is also a UK Limited Company, and one of our Principals is a Brit, and I'm perfectly happy to use the word "whilst" and drink a warm beer with you after work, if it makes you more comfortable. (Note: Americans can be even more blunt that the Dutch.) Expertise may be what matters in the end, but it's far from the only factor when closing a deal.

Stereotyping is dangerous, and as the world becomes smaller, you the technology buyer need to think more about benefiting from that which may seem foreign or "too different" for your organization. Yes, chemistry is important, but suppliers should be adept enough to adapt to your environment, and yet bring new approaches and attitudes to the project to help you be successful. Be it tennis or a vendor competition, the most appropriate mix of factors need to come together to create success, and sometimes those characteristics may not be the ones you're used to, or possessed by your fellow countryman.

As for my final take on Wimbledon: I wish Rafael Nadal wasn't injured. I'd love to see Federer break the majors record, but I'd be just as thrilled to see Roddick pull through. I don't care where Andy Murray is from, I'll cheer for him to play well, along with anyone willing to call himself a Briton.

May the best player win, wherever he's from.

A browser is a search engine?

bloem@radagio.com(Adriaan Bloem) — Tue, 30 Jun 2009 20:36:00 -0400

In a clip on YouTube, an interviewer asks passersby in Times Square what a browser is.The surprising result: many think it's a search engine.

Of course, the video has been produced by the Google Creative Lab so you could expect it to promote search engines. However, students in Rotterdam repeated the question -- and got people answering mostly the same things.

I tend to think a search engine is one of those somewhat hidden components, like the starter engine of a car. Everybody will "search" all the time, but not too many people know there's such a thing as a "search engine" behind it. Explaining the technicalities of what it takes to run it won't particularly help, either, though nobody is happy when it doesn't work well.

So if the experiment proves anything it's that the men and women in the street really don't know what's going on behind the scenes. Paradoxically, that's a sign of progress: they don't have to know in order to be able to use it, much the same way as you don't have to be a mechanic to drive a car anymore.

So before you head off on a big search, portal, collaboration, or content management project, invest the time to find out what's what and whether it's actually what's needed. After all, you may have been asked for a search engine -- when what's really needed is just a browser. And likewise, you may have been tasked with setting up SharePoint -- when what you really need is just the search engine.

Lucene can read almost anything: Lucid and ISYS team up

bloem@radagio.com(Adriaan Bloem) — Tue, 30 Jun 2009 19:19:00 -0400

A few months ago, I blogged about ISYS offering their document converter filters as a separate component. My thought was these would come in handy to add on to Lucene (which, by itself, can't actually read Microsoft Office files, let alone more exotic document types.) That would still leave you with a bit of DIY work, though: integrating the filters in your Lucene implementation.

As it turns out, Lucid Imagination had exactly that idea. The company, which offers commercial support for Lucene and Solr, is now offering it's own "LucidWorks" versions with the ISYS filters integrated. This means one of the gaps between open source and commercial search products has been bridged: with the filters, Lucene, too, can read over 200 file types.

According to Lucid, this has been one of the favorite doubts commercial vendors would cast over the open source search engine, and the move should level the playing field. However, as a customer, you should be aware that there's a couple of other things you may take for granted that are missing. Connectors to various content repositories, for instance, don't come with Lucene, not even a simple web crawler.

Still, the filters are a welcome addition, and they're certainly an improvement over what's currently available as open source. It's not just in the numbers: ask yourself how you think a converter will read a three-column Word document. You may be surprised to know that some will just go across all the first lines from left to right, then the second lines, etcetera. As always in Search & Information Access, the devil is in the details -- and knowing about these details will pay off.

The added filters aren't for free, but not exactly expensive, either. There's a 14-day trial, and you can get a subset (e.g., Microsoft Office) of the filters for as little as $3.250 for 2 years, or pay $10.000 for all of them (including those pesky legacy formats you'll discover in a distant corner of your fileserver when you least expect it.) That's still a long way off from the hundreds of thousands even a Google Appliance implementation may cost you in licensing. (Though there's no such thing as a free lunch or free beer with open source, either.)

So this is interesting news if you're considering Lucene, but what about ISYS? Aren't they selling the family silver? Well, let me wrap up this post by meandering off into history. As the (perhaps apocryphal) story has it, when the Dutch were at war with the Spanish in the 16th century, they were still selling cannons to their opponents. They figured they might as well make a profit out of it: the outcome would be determined by strategy, anyway.

Open source projects and commercial vendors, on the other hand, don't even have to be at war. And as with a Spanish Rioja or a Dutch Heineken, it's all about picking the right one for the occasion.

The Coming Acronym Crisis

kthomas@cmswatch.com(Kas Thomas) — Thu, 25 Jun 2009 13:14:00 -0400

As I talk to people in the content-technology industry (if I may call it that), I'm struck by a common thread that has begun to emerge in conversations involving roadmaps and futures. Vendors are beginning to unshackle themselves from acronyms. Let me spare you the suspense and take you straight to the disturbing punchline: I believe we are headed for an acronym crisis.

I've heard Digital Asset Management vendors say that DAM is not a good acronym any more, because it conjures a narrow, obsolete picture of the problem space. DAM platforms have grown. Offerings like MediaBeacon R3volution, MediaBin (now owned by Autonomy), North Plains Telescope, and others, are beginning to include functionalities that are, in many cases, rather ECM-like. Likewise, many WCM products -- Alterian Immediacy, PaperThin CommonSpot, and many others -- continue to incorporate more and more DAM-like features (e.g., lightbox previews, inline image editing, renditions, image metadata support, Flash previews). Some products, like Day Software's Communiqué, have such smooth integration between WCM and DAM offerings that it's hard to tell where one begins and the other one ends.

But it's not just a matter of WCM+DAM convergence (something that's been talked about, and has been happening, for a long time now). Search and Web Analytics are increasingly integral to Content Management solutions, and these technologies are, in turn, driving more personalization and dynamism into Web-facing systems. The information that comes out of all this has high business value and needs to be fed back into any number of other business applications (CRM, BI, KM, Sales Lead Management systems, etc.), lest ROI suffer. But the crisscrossing of so many technologies leads to a paradox: What do you call a system that combines features of WCM, DAM, Web Analytics, Search and Information Access, and maybe CRM as well? No one acronym seems to do the job.

All of this begs the question of whether acronyms are really that important to begin with. I think they are, actually. (Otherwise we wouldn't have so many of them -- and they wouldn't persist for so long after becoming obsolete.) As old acronyms fall into disuse, new ones emerge. CMS gives way to WCM which gives way to WEM (Web Experience Management), which in turn will someday give way to something else. Right now, though, the industry is at a crossroads. What do you call a CMS that incorporates aspects of DAM, WCM, DM, Search, Web Analytics, and Text Mining, plus (say) a few social apps? You can call it a content platform (CP), but that feels vaguely unsatisfying.

Nomenclature is important, I think. But in this case, I'm fresh out of acronyms. And for an analyst, that's embarrassing.

GSA6: Google Billions, Revisited

bloem@radagio.com(Adriaan Bloem) — Thu, 11 Jun 2009 05:09:00 -0400

Last week, I posted a highly critical comment on Google's marketing of the Appliance, version 6. My main qualm is that the hyperbole makes it very hard to understand what it actually is they're selling. What you get with a GSA is not exactly how it looks on YouTube (well, the box is, but not necessarily the internals).

Of course, in my quest to get you the real story, I'm not going to leave it at "press releases and documentation don't match up". The interesting bit is what the software is actually capable of; even more interesting is what customers are doing with it in reality.

For now, I'll zoom in on what made the headlines: the Appliance's new capability to index billions of documents, rather than the 30 million of previous version. I noted two things about this:

The Dynamic Scalibility feature, according to the version 6.0 documentation, "enables multiple Google Search Appliances to work together to scale up to 30 million documents and provide unified search results" (not billions);
Being able to index billions of documents, in general (and this applies to all vendors) is a rather meaningless statement, since it really depends on what you're indexing (I used the example of 10-digit phone numbers vs. 40mb PDFs).

Google got in touch with me to explain this, and this led to two surprises.

First of all: Dynamic Scalibility is, in fact, the feature that would enable indexing billions of documents, and this isn't a beta feature. So what about the documentation's reference to a 30 million document limit? As it turns out: this is an error in Google's documentation. (For now, the error is still in the "Guide to Software Release 6.0", but I've been told this will be corrected.) According to Google, there is no hardwired limit to the number of documents you can index using multiple machines (as long as you buy lots and lots of Appliances to do it on, of course).

Secondly, about the difference between indexing 10-digit phone numbers or 40mb PDFs: I've been told that the Appliance's hardware is carefully over-spec'ed to handle the load Google claims it can deal with. (The Dell PowerEdge R710s the vendor ships would out-perform many commodity servers). My 40mb comment was a bit of a jab: an Appliance won't index documents larger than 30mb. But as Google explained, the limit has been set so they can guarantee that when they say a GB-7007 can index 10 million documents, it can actually index 10 million of those 30mb PDFs when that's what you need to do. And to be fair, if large documents are an issue for you, you'll want to read our Search & Information Access Report product evaluations carefully, since most enterprise search products have similar limits.

In the end, of course, the proof will be in the pudding: even if the software is capable of tying together 38 appliances to index a billion documents, this may not mean you'd actually want to. What are minor issues on a smaller corpus suddenly become major problems on that scale, and I'm looking forward to seeing how real enterprises are faring in deploying a cluster of GSAs for such high volumes.

And if anything: you still shouldn't believe the hype. Google's "billion document index" headline was syndicated across hundreds of news sources before even Google itself found out its documentation contradicted this. You'll want to be sure to get your information from a reliable source.

Google Search Appliance v6: don't believe the hype

bloem@radagio.com(Adriaan Bloem) — Tue, 2 Jun 2009 13:54:00 -0400

The release of a new major version of the Google Search Appliance usually creates lots of excitement, but that excitement fades quickly once people start to use the machines for real. Even many of the Google resellers I talk to admit to some disappointment: first highly anticipated features are finally introduced, and then they turn out to be rather DIY.

I don't want to sound like a broken record -- see Google Search Appliance: small step in technology, giant leap in marketing or The Emperor's New Box. The point with many of the Google Appliance's new features is that typically, they're not actually on the box itself, and quite often, that means you start loosing the convenience of having a plug-and-play appliance.

So pardon me for being skeptical of what the new version 6 of the yellow server will actually bring to the table. Try to line up the several lists of new features: the PowerPoint presentation I saw was slightly different than the "New!" list on the product page. The YouTube announcement mentions cross-language-enterprise search, which isn't a feature but an experimental project. Actually, looking at the version 6 documentation, quite a few of the new features turn out to be still in beta. Many of the announcements mention "early binding" security as a new feature, but I haven't been able to find this in the documentation. The same goes for many of the other capabilities you'll hear buzzing about: they're very hard to trace back to actual features in the new version 6.

Perhaps the most hyped novelty is scaling, or as Mountain View likes to call it, (GSA)ⁿ. As the Google Enterprise blog gushes, "When we tested it out, the product manager was pretty excited about all the new features and search power. He was used to hearing about the millions of docs we could handle – but this time we were going to push it to a new realm: billions." They've even made a video that shows the product manager being all excited. I can imagine him being excited -- like many others, Google charges per indexed document. But a flat claim of "billions," with "less than five server racks!" means very little to me. Billions of what? 10 digit phone numbers or 40mb PDFs? (I can go on for hours on why this is quite meaningless, since it all depends on the exact circumstances, but suffice it to say that I've already seen several vendors demonstrate billion-document-indexes on a lot less than five server racks).

However, I was pleased to see that this is one of the features that's actually on the box itself in version 6. Well... "Dynamic Scalability" is, at any rate, which "enables multiple Google Search Appliances to work together to scale up to 30 million documents". So how do you get from 30 million to billions? I presume by using "Distributed Crawling", since this "greatly increases the number of documents that can be crawled". That's a beta feature, though, which can only lead me to conclude that the theme of this release should actually be (GSA)^beta (pronounced as "to the power of beta").

I tend to think that in every market, there's room for Rolls Royces, Volkswagens, and everything in between. But it makes very little sense to take a Volkswagen and turn it into a Rolls Royce. Thankfully, Volkswagen understands this. Maybe Google should, too.

When customers are unreasonable

tbyrne@cmswatch.com(Tony Byrne) — Wed, 27 May 2009 19:22:00 -0400

At CMS Watch, we're advocates only for you, the technology customer. But sometimes the customer is unreasonable. In my experience, "bad" customers most typically cause problems around schedules and scope, more so than cheap-skating the vendor.

Nonetheless, this video nicely captures how customers can go too far demanding more value than what they're willing to pay. It's funny. (Hat-tip to Jennifer Mayne Hoppe and Heidi Strom Moon.)

But let's talk about the world of software and implementation services -- which alas is not like ordering dinner, or buying a DVD, or getting your haircut.

If this clever video was about the real world of software sales, then...

There would be no prices on the restaurant menu
The DVD cover price would include the box only ("Oh, so you wanted the disk, too?")
Highlights could only be acquired as part of a larger "make-over suite" of bundled services

Our research is designed to help you save money, and part of what we explain to you is how to negotiate a better deal. At the end of the day, though, your suppliers are in business too, so the best relationships engender transparency -- but also allow for both parties to profit.

Wolfram Alpha search engine: just the facts

bloem@radagio.com(Adriaan Bloem) — Mon, 25 May 2009 14:53:00 -0400

After months of high anticipation of what this "Google killer" would bring, Wolfram Alpha was finally opened to the public a week ago.

It's common practice in this day and age to compare any new public search engine to Google. Or more to the point, to debate whether or not the incumbent will be able to snag a piece of the giant's share of searches, currently just over 70% of the US market, but in most of Europe almost monopolizing at over 90%. This was the main point with Cuil, and it will be the main point with Microsoft's upcoming new web search engine, codenamed "Kumo".

Wolfram Alpha, of course, has to be different. And in this case, it actually is; so much so that it makes very little sense to compare it to Google. It's a "computational knowledge engine," which tries to "understand" your query (not so much in a natural language way, though it does try, but more as a mathematical equation) and then calculates the answer based on a corpus of "curated facts."

If you must compare it, look at Google's "reference tools," e.g. queries like "10 dollars in euros" (currently, about 7.26). Wolfram Alpha certainly does a better job at this (including nice graphs of the rate over the last year or last ten years). It will also, for instance, "compare York to New York" for you (and thankfully, it'll show you how it has parsed your request). Think of it as an excellent way to query sources like the CIA World Factbook or Wikipedia and perform comparisons.

Playing around with it, you'll also run into its limitations quite quickly. You can ask it the average temperature in San Fransisco or the average temperature in The Netherlands, but it won't compare the average temperatures in San Fransisco and The Netherlands. You can ask it to divide 12 by 3 and calculate the result as Dollars to Euros, and you can ask it for the "currency of The Netherlands." But you can't ask it what "4 in US currency is in The Netherlands currency."

In short, while it allows for very complex mathematical queries, please don't ask it to check for more than two facts about facts at the same time. And if you're not looking for facts, you're completely out of luck: it can show you the color purple (the book), but you'll learn little more of substance than that it was first published in 1983 and won a Pulitzer.

It's tempting to digress on the roots of Wolfram Alpha and talk about Stephen Wolfram, his Mathematica software, and his book "A New Kind of Science", which attempts to answer, well, the question of life, the universe, and everything. But to be honest, this is probably as helpful as an explanation of Bayes' theorem is to understanding the usefulness of Autonomy IDOL. The main difference is probably that Stephen Wolfram, unlike Thomas Bayes, is alive and well, and his ego is an important driving factor behind Wolfram Alpha.

There's certainly a use for the engine, and you may want to bookmark the site for that reason. You may even want to use the API for this functionality once it becomes available. But "computational knowledge" has its limitations. Wolfram Alpha will, in fact, by using its principles of Mathematica and NKS, compute the answer to the question of life, the universe, and everything. The answer is 42. I'll leave it up to you to decide whether this just is the wrong question to ask, or if the answer is just really unhelpful.

Search, query syntax, Google and Starbucks

bloem@radagio.com(Adriaan Bloem) — Wed, 20 May 2009 15:33:00 -0400

There's too much here to write up in one blog post, so I'll pick one of the cherries for you here. Last week's Enterprise Search Summit in New York ended with roundtable discussions -- which meandered around several topics, and then down the escalators into the hotel lobby.

I was discussing querying a search engine with Daniel Tunkelang (Chief Scientist of Endeca), standing across the street from one of the ubiquitous Starbucks. He told me search phrases are much like coffee orders: if you don't go to Starbucks often, you'll hesitantly ask for a large coffee, with milk, maybe skim milk, and you'll say please. If you go there every day, you just go in and say "venti skim milk latte," pay, and leave with your coffee.

When we use Google on the web, we're used to the same kind of formulas -- we know Google's query language because we've grown accustomed to it, not because it's particularly good or bad at understanding what we want.

Endeca is doing research in parsing user's queries into meaningful commands for a search engine (as are many others), and it'll be interesting to see what they come up with. But there is an useful insight here that has a broader appeal: while public web search engines (like Google) have an interest in making the parsing seem easy, within the enterprise there's no reason to hide what processing has been performed on your original query.

As I walked into Starbucks the next morning for my daily dose of caffeine (sadly lacking my Lavazza Qualità Oro when traveling), I realized it once took me several attempts explaining this as "a large espresso," "an espresso with three extra shots," or "a double doppio." I broke the code because I could hear the order go to the barista as a "quad."

Maybe you should let your users see what you're actually ordering the search engine to serve up, as well. Show what you're expanding, removing, or translating. And don't forget that "no results" isn't necessarily a bad thing -- as long as you're sure there really are no results to display. It's more helpful to be straightforward than overly polite!

Search, Portals, and SharePoint at Interop-Vegas

tbyrne@cmswatch.com(Tony Byrne) — Mon, 11 May 2009 15:04:00 -0400

Next week you can find Alan and I at Interop 2009 in Las Vegas. We'll have a booth there to show some of CMS Watch's latest research, so feel free to stop by and say hello.

On Wednesday, we're also co-teaching an intensive, day-long workshop on contemporary enterprise information management. The morning session covers "Accessing Enterprise Information via Portals and Search Technology." Then after lunch we tackle "Evaluating SharePoint in the Enterprise."

If you're interested in Portals, Search, and SharePoint, I encourage you to sign up. There's a lot happening in all three domains, and we're going to see if we can sort it all out in one action-packed day. Hope to see you there!

JBoye 2009 conference wrap-up

tbyrne@cmswatch.com(Tony Byrne) — Fri, 8 May 2009 09:26:00 -0400

I had the pleasure of participating in the JBoye 09 conference in Philadelphia, USA earlier this week. For those who missed it, here are some interesting tidbits, in no particular order.

Lou Rosenfeld talked about the importance of cross-pollinating the work of information architecture, content management, and user experience (money quote: "CM-UX = money down the toilet"). But the meat of his keynote revolved around how to reconcile the different approaches of web analytics and user experience. Web analytics tries to answer "what." UX tries to answer "why." Of course you need to ask both, but when you ask them together, you can get some very revealing answers.

Jim Hobart gave a high-speed tour of different approaches to navigating complex hierarchies, based on actual customer testing. A sprinkling:

Avoid three levels of trees in casual user interfaces
Avoid cascading menus or menus that require fine-motored mousing skills (we have to fix ours!); indent, or use single-level drop-down instead, with lots of signaling
The ideal wait-state for a mouseover pop-out is 1250 to 1800 milliseconds, depending on the size of the target area

From the annals of making-sure-your-SharePoint-consultancy-knows-what-the-hell-they're-talking-about: Dorthe Jesperson of JBoye shared an anecdote from a major Danish enterprise whose Microsoft partner told them repeatedly (!) that, "it's impossible turn off MySites." Ahem. Actually it's considered a best practice by many large enterprises to disable MySites upon initial install and then selectively turn them on after addressing technical and governance issues. See our SharePoint research for more details.

The ever thought-provoking Martin White and Bob Boiko set off some great discussions about whether we should call ourselves "information managers" rather than "content managers." The debate is of course partly over semantics (personally, I think Information = Content + Data, but some people find that too trite), and partly about better marketing the importance of what we do within the enterprise. Is "information manager" a more impactful title? ...I don't think so. But then, what is...?

On a related note, I had the pleasure of moderating the final "town hall debate" between Dana Hallman of the US Treasury Dept and Jerry Boyle of Thermo Fisher Scientific. The following resolutions were put forth to the audience, who voted on them after some lively sparring:

"Face it: Your privacy is long gone" -- passed

"We need to stop whining and start doing" -- passed

"I've seen the future and it's SharePoint" -- failed

"Communications Directors should call themselves Information Managers" -- failed

"LinkedIn will soon become an important enterprise information management platform" -- failed

In sum, the conference was a super learning experience. If you care about web and web content technologies and practices, this is a great event.

Search in New York next week

bloem@radagio.com(Adriaan Bloem) — Wed, 6 May 2009 18:27:00 -0400

I'm excited to be headed over to New York next week. The Enterprise Search Summit 2009 brings together a veritable who's-who in search, and I'm looking forward to attend an event where search is the featured topic (and not just another parallel track on the program).

If you're going, be sure to say hi to us. We love to hear real-life stories about search from end-users in the enterprise and exchange thoughts with you. I'll be doing a presentation (my 2 cents' worth on pragmatic ways to implement search), but I'm really attending to listen (rather than just talk).

If you haven't decided yet, it's not too late. You can attend the keynotes for free, and better yet, you can get a $200 discount if you sign up using this link. Hope to see you there!

Join us at Internet World UK

tregli@cmswatch.com(Theresa Regli) — Fri, 24 Apr 2009 13:22:00 -0400

When I think of Earls Court, London, the first thing that comes to mind for me is the many Led Zeppelin concerts I grew up listening to that were recorded in this hallowed venue. Next week Earls Court will take on a different sort of ambiance, to be sure, for Internet World UK, the largest expo of its kind in London.

Join me and my colleagues Tony Byrne and Alan Pelz-Sharpe (who are almost, but not quite as cool as, Jimmy Page & Robert Plant) as we host sessions about Social Software, Web Analytics, Findability, and DAM. We'll also have a stand featuring our now-infamous "content management therapy" couch. Come by this free show, spend some time with us, and share your content management challenges.

Vivisimo - still searching for real social marketing

bloem@radagio.com(Adriaan Bloem) — Tue, 21 Apr 2009 04:48:00 -0400

Meet Stan, the everyman, who'll try to sell you on Vivisimo Velocity enterprise search software. Stan stars in videos, explaining the wonders of enterprise search, writes a blog, can be friended on Facebook, and followed on Twitter.

While I appreciate attempts at explaining what enterprise search is, or can do, I'm not sure Stan is going to help. It may just be me, but when I hear "Meet Stan", I'm expecting Eminem ("Meet Stan. After meeting a young girl at a rave party, things start getting hot and heavy in an upstairs bedroom"). That must be a different Stan, though; in Vivisimo's world, all men wear mustaches. (And you recognize "hot women," like Anne, because, well... they have no facial hair.)

So interestingly, there isn't really a lesson here about enterprise search. Rather, this is an example of how hard it is to use social software for marketing purposes. Vivisimo, for one, seems to have confused "viral" with "cheesey."

Stan does reply to Twitter comments, but that has mostly taught me he likes microbreweries. It's not the kind of interactive marketing that will engage an audience; you will need to take the people you're addressing a bit more seriously. Stan is talking to us -- rather than talking with us.

On the upside, at least Stan doesn't speak in Comic Sans. But if you're thinking of interactive marketing, or using social software to reach your customers, you should probably think more about what content would attract visitors, and how you're going to interact with them, rather than having an agency design something that ends up as a rather static brochure avatar.

Read me that file so I can index it, please

bloem@radagio.com(Adriaan Bloem) — Wed, 8 Apr 2009 15:20:00 -0400

One of those easy-to-overlook but important details of a search engine: will it actually read your files? You may be interested in Lucene, but you'll have to find a way to feed it Office documents and PDFs.

Search engines don't actually directly index the Word document or PDF, they index text. This is where document filters come into play. These do their best to get the text from the file (and usually some metadata, such as an "author" field). If you've ever tried to open some exotic document format in a plain text editor (i.e., Notepad, or VI) you'll understand this can be far from trivial: many of these formats aren't very straightforward.

The problem isn't just trying to find the text, there are quite a few complications: reading across two or three column layouts; what to do with footnotes; or what to index, period. Spreadsheets are troublesome, but what do you make of images, audio, video? And for many scenarios (like indexing a file share) there will be exotic file types to deal with. (I recall the comments at a municipality once: "But we don't have any exotic file types". Three months later, a full crawl unearthed a stack of CAD/CAM files that were vital for planning). To make matters worse, file formats change with the software versions that come out (will the converter read Office 2007 or just Office 95?).

Since it's complicated to build and maintain good filters, most vendors buy them off-the-shelf. As I've talked about before, the market has been cornered by Oracle (with the INSO filters) and Autonomy (with the KeyView filters). Almost all the search engines out there use either Oracle's or Autonomy's converters. A notable exception is Microsoft, which has its own standard for this, IFilters. But IFilters are of varying quality, they don't always work with every Microsoft software product, and you may very well have to build a custom filter yourself for some ancient or rare software.

And there's ISYS -- probably the only vendor we cover in our Search & Information Access Report that has developed converters for over 200 document types entirely by themselves. (Even Oracle and Autonomy didn't really build filters themselves -- they bought the companies that produced them).

It makes sense, then, that ISYS now tries to bank on that hidden capital. The vendor announced last week it's releasing its File Readers as a separately available product. It'll be interesting to see these show up in Lucene implementations (and in content management systems embedding search). More options means more choice. Black may be the fastest drying paint, but maybe you can now have that Model T in purple again.

Listening to the Pogue

tbyrne@cmswatch.com(Tony Byrne) — Tue, 7 Apr 2009 00:27:00 -0400

That would be David Pogue, New York Times technology columnist and author of the "Missing Manuals" series of books. He'll be delivering the first keynote at JBoye 09 in Philadelphia next month. Even better, he's teaching a 3-hour (!) course on "Google: The Missing Manual." Both promise to be as entertaining as they are informative.

Hope to see you in Philly in any case. (BTW, you can get a 20% registration discount using the code cmswatchphilly.)