Search engines & IDX (note regarding renaming)

For the four or five of you actually following the lengthy discussion on search engine idexing of IDX sites, I wanted to give you a heads up that I have renamed the posts on that topic to shorten the names (removing the academic colons, collapsing the two levels of title into one, etc.), with the goal of making the posts easier to find in the post history for MLSTesseract.

Sorry if this causes any confusion.
-Brian

Search engines & IDX Part V: Is search engine indexing legal?

(Warning: possibly mind-numbingly boring and long legal content to follow. And I'm not even giving legal advice here.)

When I posted regarding indexing and search engine audiences and objectives, I mentioned that I could provide a short exposition of how copyright law sees search engines. Only Paula Henry had the courage to ask a lawyer to discuss a legal topic in her comment on that post. With only one taker, I thought maybe I'd pass on the chance to explain, but then I got thinking how this might fit in with later posts.

My knee-jerk instinct when folks asked, "Is it fair use if Google indexes listings on an IDX site?" was to say, "Of course. We love web search engine indexing." As it happens, when I began to think over how the prevailing case law treats web search engines in the copyright context, I started thinking I may have been a little hasty.

So, here it is. Kelly v. Arriba Soft Corp. (2003), from the Ninth Circuit Court of Appeals (the federal court that hears appeals from California and a couple other western states), is the leading case in analyzing copyright fair use in the search engine context.

Facts from Kelly

(For you lawyers, you can find the Kelly opinion at 336 F.3d 811 (9th Cir. 2003).)

Plaintiff Leslie Kelly was a professional photographer. Some of his images appeared on his web site and other sites to which he had licensed them. Arriba Soft Corp. operated an Internet image search engine. (It's name has changed and it's now available at http://www.ditto.com/.) Arriba's web crawler cached full-sized copies of the images onto Arriba's server. (See discussion of indexing and caching in previous post.) It then created 'thumbnails' (smaller, low-res versions) of them. It allowed visitors to search for images. Arriba Soft then displayed thumbnail versions of matching images. According to the court:

by clicking on the 'Source' link or the thumbnail from the results page, the site produced two new windows on top of the Arriba page. The window in the forefront contained solely the full-sized image. This window partially obscured another window, which displayed a reduced-size version of the image's originating web page. Part of the Arriba web page was visible underneath both of these new windows.

All of this detail, it turns out, was not necessary to the court's decision, as the court addressed only the question of Arriba's display of the thumbnail images. A technicality in legal procedure prevented the court from considering the question of displaying the larger-scale images.

Kelly never gave Arriba permission to copy his images and objected when he discovered the copying. Based on his complaint, Arriba took the images down and blocked indexing of the sites where his photos appeared, but he sued anyway. Kelly claimed violations of his exclusive rights to display, reproduce, and distribute his copyrighted works. The trial court granted Arriba Soft's motion for summary judgment (basically dismissing the case), both with regard to display of the thumbnails and the full-sized images, based on the fair use doctrine. Kelly appealed to the Ninth Circuit.

The Kelly court's basic rules of copyright law

Here are the basic rules of copyright law, as the Kelly court laid them out in its opinion.

• The owner of a copyright in a work has the exclusive right to reproduce, distribute, and publicly display the work.
  
• For Kelly to establish that Arriba Soft infringed his copyrights by reproducing his work, he had to show that he owned the copyrights and that Arriba copied the works. (As to the thumbnails, Arriba conceded that it had copied the images and that Kelly owned the copyrights in them.)
  
• If the plaintiff establishes that the defendant has copied the plaintiff's copyright-protected work, the defendant may escape liability if it can show that the copying is "fair use."
  
• Fair use is an exception to the general rule of copyright law, designed to permit creativity and productive output that rigid application of the copyright law might otherwise impede.
  
• Whether a use is a "fair use" depends on the weight given to and interaction between four factors. The court can consider other factors, but it must consider the four identified in the Copyright Act.
  
• The four "fair use factors" are 1) the purpose and character of the use, including whether the use is of a commercial nature or is for nonprofit educational purposes; 2) the nature of the copyrighted work; 3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and 4) the effect of the use on the potential market for or value of the copyrighted work.
  

Factor one: Purpose and character of the defendant's copying

First, just because Arriba's use was commercial, it did not necessarily lose on this factor. Quoting an earlier case by the U.S. Supreme Court, the Ninth Circuit said the central purpose of this factor is to determine

whether the new work merely supersede[s] the objects of the original creation, or instead adds something new, with a further purpose or different character, altering the first with new expression, meaning, or message; it asks, in other words, whether and to what extent the new work is transformative. (BNL's emphasis)

The Ninth Circuit said that the "more transformative the new work, the less important the other factors... become." As we shall see, this is an important observation. The court compared Kelly's purpose in the works ("intended to inform and to engage the viewer in an aesthetic experience") with Arriba's (intended "as a tool to help index and improve access to images on the internet and their related web sites"), and found Arriba's copying to be highly transformative. The court found that this factor weighed "slightly" in favor of Arriba, with the transformative use outweighing its commercial nature.

Factor two: Nature of copyrighted work

Some works warrant greater protection under copyright laws than others. For example, works of art generally get higher degrees of protection than compilations of facts. Because photographs of the kind that Kelly created are close to the "core of intended copyright protection," the court found this factor was slightly in Kelly's favor.

Factor three: Amount and substantiality of the portions used

Generally, the more that one copies of the original work, the more likely this factor will weigh for the plaintiff. But where, as here, copying the whole image is necessary for the transformative use, the court found this factor did not weigh in favor of either party.

Factor four: Effect of the use on potential market or value of the copyrighted work

This factor considers the extent of market harm caused by the defendant's use and the likely impact on the market for the original work that defendant's conduct would have if engaged in by many others.

The key here is that the thumbnail images on Arriba's site (the only ones considered in this opinion) were not in any way a substitute for any use that Kelly was making of his own images. They were too low-quality to be of use in any size larger than a thumbnail. The court noted, though, that this might not always be the case with photos. (In fact, in a later opinion, the Ninth Circuit would acknowledge that this factor might weigh against the defendant if it indexed images from a source that made a business of marketing images in thumbnail form.) It also noted that low-quality copies of other types of media (e.g., text, audio, etc.) might not always result in a win for defendants on this factor, either.

The court determined that this factor went for Arriba.

The tally of factors favors Arriba

The court noted that one factor went slightly for Kelly (factor two), one was neutral (factor three), and two went for Arriba (factors one and four) – the tally thus favored Arriba. It's interesting that the court took this approach for two reasons: First, the court seemed to use the transformative nature of the use (factor one) to make amount and substantiality (factor three) a 'push'; second, the court's analysis of the effect of the use (factor four) found Arriba's transformative use did not compete with Kelly's uses.

In fact, I think the very transformative nature of Arriba's use was the deciding "factor" in three out of four factors in the fair use test.

Is the current dust-up like Kelly?

Remember, my knee-jerk instinct when folks asked, "Is it fair use if Google indexes listings on an IDX site?" was to say, "Of course. We love web search engine indexing." But think about the Kelly court's focus on the transformative nature of Arriba's use and compare it to the IDX search engine situation. If a web search engine indexes the contents of an IDX search engine, is it transforming the content? Isn't it just copying the whole thing, possibly slightly transforming it, and then replicating the utility of the original? Don't three of the four factors now go against the defendant/web search engine?

I'm curious what you think. (You don't need to be a lawyer to weigh in on this.)

Copyright licenses and ROBOTS.TXT

But even if a web search engine indexing an IDX search engine is not fair use, I think the web search engine has one more defense – permission. In copyright law, as in real estate, giving someone permission to use your property is called "granting a license." (The term is less common in real estate law than in intellectual property – but it has a long history in 'dirt law.')

Having a license is a defense to a copyright infringement claim. If you say I copied your work, and I can prove you gave me permission, you lose your copyright infringement claim against me.

Licenses can be implied: that is, the copyright-holder can behave in such a way as to suggest that she is granting a license (even if she never actually says, "You may copy this"). Generally, she can revoke an implied license at any time.

I expect if an MLS sued Google over indexing an IDX site, Google would claim that that the IDX sites it indexed had given it an implied license to copy for at least two reasons:

1. By creating static pages of listings and detailed sitemaps, the IDX site was "index-fishing," the only point of which is to be indexed. Thus, the site owner cannot complain when the indexing actually happens.
   
2. Any web designer worth his/her salt knows that you can put a "ROBOTS.TXT" file on a web site or portion of a site to keep search engines away. If the owners of these IDX sites did not want them indexed, they could easily put the file in place, expressly revoking any implied license.
   

There's only one problem: Just like in real property, one cannot grant any more rights in intellectual property than one actually has. The broker who displays listings on an IDX site has a license from other listing brokers and the MLS to display their listings on the displaying broker's IDX site. But the broker does not have the right to license that content to other sites, including web search engines.

But listing brokers can do what they want, right?

Here's the key (and I think this is responsive in roundabout way to a comment Victor Lund posted a few days back): The broker operating an IDX site could certainly give a license to copy its own listings to web search engines. In fact, I don't think that NAR's IDX policy (or the MLS rules promulgated under it) prevent a listing brokerage doing whatever it likes
with its own listings.

Consequently, if a listing broker wants to put up an IDX site and index fish using its own listings as bait, I don't think any MLS is likely to complain. I don't think that would require an NAR policy or rule change at all. Just don't create static pages and sitemaps for the listings of your competitors.

-Brian

Search engines & IDX Part IV: "IDX index fishing"

IDX sites are not naturally prone to be indexed by web search engines like Google. Here's why:

Typical IDX search engines 'draw' pages of listing data dynamically or 'on the fly.' When the consumer enters her criteria on the search page, the IDX search engine goes into the database and pulls the data for matching listings. The IDX search engine then formats the listing data for display on the IDX web page. Before the consumer's search, there was no web page displaying the listing at 123 Elm Street. It was only when the consumer executed her search based on criteria that included 123 Elm Street that a page with information about the listing was drawn.

When Google's web crawlers encounter a web page for a typical IDX search engine, they index the contents of the IDX search engine's search page. But the Google web crawler does not execute searches on the IDX search engine, and consequently, the listings on the IDX site are not indexed at Google.

Clever brokers and technologists have found a way around this problem. They have their IDX sites draw a static page for each of the listings in the IDX database, and they may include links to all those pages from a 'site map' page on the IDX site. So every day, the IDX site creates a slough of pages, one for every listing in the IDX database. The pages are drawn in such a way as to be easy for Google to index. The display in question may be nothing like what is drawn for the consumer when she executes her search, and in fact, most of those pages will never be seen by any visiting consumer. That's not important to the site, though – what's important is web search engine optimization.

The first time I saw this practice, I was impressed by its cleverness. I was shown it by an industry consultant. The consultant had me visit the web site of ABC Realty and look at one of the listings of its agent Nancy Smith at 123 Elm St. (All names and addresses are changed in this example.) The consultant then directed me to Google. He said, "Imagine you saw the 'for sale' sign, remembered it was on Elm Street, and that the listing agent was Nancy Smith." So, I typed "nancy smith elm street" into the Google search engine. The top-matching entry (of the relevancy matches – there were no sponsored links) was for a page on Clever Realty, Inc.'s website. I clicked on the link, and up popped Nancy's listing, but on Clever Realty's site, not Nancy's site or her broker's. ABC Realty was credited as the listing broker (as required by the IDX rules) but all the contact information on the site was for Clever Realty and its agents.

This is what I call "IDX index fishing." And it is this practice that makes Google indexing of IDX valuable to the brokers who operate the sites (and the vendors who sell them their technology). To see how it works, try it with one of your own listings: Go to Google; for search criteria, use your name and the street name of one of your listings.

I cannot think of any reason for an IDX site to offer a static page for each IDX listing and a site map for those pages except to fish for web search engine indexing. Consumers do not browse listings by listing number or even street name (usually).

Note that IDX index fishing is not illegal or improper under the IDX rules of most MLSs, as long as the resulting static pages comply with the MLS rules. Note also that IDX sites are not the only sites that do index fishing – Realtor.com and most national aggregators also do it.

Let's look at another example:

1. I was looking for a cabin/lake home up north in Minnesota. I knew that I was interested in lakes around the town of Moose Lake. I found the name of Majestic Pine Realty, a Moose Lake broker, via a banner ad on a non-real estate site. So I went to Majestic Pine's site at http://www.majesticpine.com/ and viewed listings.
   
2. I found a listing I liked - http://www.majesticpine.com/viewListing.aspx?LID=237. I noted that it was on Soper Lake. I don't know Soper Lake, so I wanted to find out more about the body of water.
   
3. I googled "Soper Lake mn".
   
4. Here's where things get interesting. The first three matches of my Google search were these:
   
   
   1. Soper Lake Fishing in Carlton County, Minnesota Fishing Works
      
      

      Soper Lake is a lake in Carlton County in the state of Minnesota. The latitude and longitude coordinates for this lake are 46.4272, -92.4916.
      www.fishingworks.com/lakes/minnesota/carlton/holyoke/soper-lake/ - Cached - Similar pages -
      

   2. 1939 Soper Lake Ln, Holyoke, MN, 55749 - MLS ID#145561 - Single ...
      
      

      1939 Soper Lake Ln, Holyoke, MN, 55749 - Single Family Home - MLS ID #145561. 2 Bed, 1 Bath. Real estate listings and homes for sale on REALTOR.com®
      www.realtor.com/realestateandhomes-detail/1939-Soper-Lake-Ln_Holyoke_MN_55749_1108124784 - Cached - Similar pages -
      

      Holyoke, MN, Real Estate Listings and Holyoke Single Family Home ...
      

      Photo of 1939 Soper Lake Ln, Holyoke, MN 55749. $149900. 2 Bed, 1 Bath, 984 Sq Ft on 5.67 Acres. Property Type: Single Family Home. Save Listing ...
      www.realtor.com/realestateandhomes-search/Holyoke_MN/type-single-family-home - Cached - Similar pages -
      More results from www.realtor.com »
      

   3. 1939 Soper Lake Ln, Holyoke MN 55749, MLS #145561, Weichert.com
      
      

      Features of this property include 1 level living, tongue and groove pine throughout, open kitchen, vinyl energy windows and attached garage.
      www.weichert.com/24732296/ - Cached - Similar pages -
      

5. I was gratified to find the first link, which told me a little about the lake I was interested in. But I was intrigued by the next two matches – one for Realtor.com, and one for the IDX site of a Weichert office in Minnesota, but both referring to my listing (actually, to Majestic Pine's listing)...
   
6. I then went to Google Images and did the same search (looking for pictures of my lake). Here are the first results:
   

1939 Soper Lake Ln, Holyoke, MN, ... 310 x 233 - 25k - jpg www.realtor.com [ More from p.rdcpix.com ]		1939 Soper Lake Ln 320 x 240 - 16k - jpg www.weichert.com [ More from images3.weichert.com ]		1939 Soper Lake Ln 320 x 240 - 24k - jpg www.weichert.com

7. Again I got Realtor.com and the Weichert site. This time I decided to check out the links. I clicked on the Realtor.com link and ended up with a view of the listing with the listing broker's name and phone number, but no link to back to the listing broker's page (I guess you have to pay extra for the site the Realtor family owns to link back to the listing broker's site). In fairness, there was no link to anyone else's site, either. On the Weichert page, I saw the name of the listing broker (pursuant to IDX rules), but all contact info and links go to Weichert.

8. I searched "soper lake mn" on bing and got similar (though by no means identical) results.

9. I decided to try one further check. What if I remembered just the town and number of bedrooms of my original listing? So I Googled "Holyoke mn 3br" – the listing I'm interested in comes up, displayed on Realtor.com (first matching result) and the Weichert site (third matching result.)

I think we've established that some brokers with IDX sites are intentionally using the listings of other brokers in IDX to fish for indexing by Google and other web search engines. In fact, I think that's the only reasonable explanation for the static pages and site maps on some of these brokers' sites.

The next question, then, is "So what?" If Realtor.com and third party sites like Trulia and Zillow can fish for web indexing, why can't broker IDX sites? I guess the answer depends on what you think listing brokers' rights are and what their expectations are about how other brokers will use their listings in IDX.

-Brian

Search engines & IDX Part III: Search engines indexing, audiences, and their objectives

Indexing

Search engines can return results quickly because they 'index' the content in the database. When a consumer searches on a broker's IDX site for a listing number that she saw in a print ad, the displaying broker's search engine finds the matching listing immediately. If the database were not indexed, it would have to do the equivalent of checking every listing record, asking, "Is this record #1234567?" Even at computer speeds, that would take a long time. The index speeds this process.

IDX and MLS consumer-facing search engines are usually indexed based on fields of data: Listing number, bedrooms, bathrooms, etc. This kind of indexing is common in relational databases. This works well, because the visitor to an IDX or MLS site can usually answer questions about particular data fields. E.g., how many bedrooms, bathrooms, what listing price range, etc.?

Web search engines like Google work differently, indexing whole web pages. They note things like the frequency of a word on the page, its propinquity to other words, and much more, using complicated indexing algorithms. That's why you get much better results when you provide multiple words in your search. "Hepburn movie leopard" is more likely to get the answer you are looking for than "leopard," or "leopard movie." (The answer I wanted was "Bringing Up Baby.") The web search engine finds pages that have more of your words on them. Proximity of the search terms matters, too. That's why when you search "marcy holmes," a page about the Marcy-Holmes Neighborhood Association comes up more readily than a page that refers to Marcy Smith's affection for Sherlock Holmes.

It's important to understand that for Google to perform the sophisticated types of indexing it does, it must 'cache' (or retain a copy of) pretty much every web-page that it indexes. This copying of copyright-protected content from web sites might raise copyright law issues, but the Federal courts have generally held that search engine indexing of the kind that Google does qualifies for treatment as 'fair use' under the copyright laws. (It's not clear whether ALL search indexing would meet the 'fair use' test, which is a multi-factor balancing test. Some cases have found that Google's indexing of images, for example, might not be fair use if it impairs the copyright-holder's ability to commercialize its work. (Comment if you want more of a copyright law exposition on the search engine issue.)

But how you index your search engine may depend a great deal on your target audiences and their objectives.

Search engine audiences and their objectives

Search engines can be targeted at broad audiences or narrow ones; and they can be 'destination' sites or 'conduit' sites. A search engine can be designed to redirect visitors to other sites that are the source of the indexed information – a conduit site – or it can be designed to present the information directly to the visitor without resort to other sites – a destination site. Search engines vary in their audiences and objectives:

• Google is a conduit site with a broad audience. Web users of all kinds visit looking for information on topics of all kinds, and Google redirects them to the sites that are the sources of its information. (This is a generalization – Google does offer services that don't work quite this way.)
  
• IDX search engines are destination sites with a moderately targeted audience. They attempt to attract and retain the interest of consumers interested in real estate in the geographic area(s) where the broker practices (still a pretty broad group); they do not 'refer' visitors to listing brokers' web sites, but instead attempt to build relationships with them through rich content. (More on this in another post.)
  
• Real estate aggregator search engines generally have the same audience as IDX search engines (though perhaps with expanded geography). The extent to which they are destination or conduit sites varies greatly. For example, visit Realtor.com and try to get to the listing broker's web site from the average listing. Compare that to Trulia or Googlebase. (I presume it gets easier to get to the listing broker's site if she has paid Realtor.com for some kind of 'enhanced' service.)
  
• MLS consumer-facing search engines have roughly the same audience (and scope as we noted before) as IDX search engines, but they vary in the extent to which they are destination sites. The Houston Association of REALTORS site, HAR.com, for example, has very high traffic numbers, probably because it is designed to keep consumers on the site. (Again, try to get to the listing broker's page for the listing you are viewing on HAR.com.) Other sites may function more as conduits.
  
• The online legal research service Westlaw is a destination site with a relatively narrow audience. It indexes court opinions (among other things), principally for use by lawyers who are performing legal research. Westlaw's goal is to sell the password-only index itself (and certain ancillary services). It does not redirect its customers to the web sites of the courts whose opinions it indexes; it has cached, indexed, and annotated copies of all the opinions and delivers the whole kit and caboodle.
  

Next, we'll look at some SEO efforts of brokers operating IDX sites to enhance their appearance on web search engines like Google, including the practice I call "IDX index fishing."

-Brian

Search engines & IDX Part II: Search engine scope and data sources

Broadly speaking, a 'search engine' is something that permits a visitor to a web site to seek content in a database hosted on that web site. It is the engine that powers the visitor's search. In this broadest sense, every web site that lets a consumer search through a database for something is a 'search engine.' Several characteristics help to distinguish various types of search engines from each other, including their scope, data sources, methods of indexing, and their target audiences and audience objectives.

Search engine scope: Each search engine has a 'scope.' In the case of Google, for example, the scope of the search is all web pages (and a few other things, depending on where you are searching on Google). I'll refer to Google and its ilk as 'web search engines.' In the case of Realtor.com, Zillow, and Trulia, which I'll call 'real estate aggregator search engines,' the scope is real estate listings from participating brokers and MLSs. The scope of a local broker's IDX site, an 'IDX search engine,' is the IDX listings from the MLS(s) where that broker is a participant. The local MLS's consumer-facing search engine usually has the same scope as the local broker's IDX site.

Search engine data sources: Every search engine has to have a source for its data. In the case of IDX search engines, the source is the MLS's IDX program, consisting (usually) of limited fields relating to (usually) active listings of brokers in the MLS, provided neither the broker nor the seller has opted out of IDX.

A real estate aggregator search engine's data source is usually (a) listing brokers/salespeople uploading their own listings; (b) MLSs or listing syndicators syndicating listings to aggregators on behalf of listing brokers/salespeople; (c) other aggregators that got listings via one of the first two means, sharing listings under a business relationship between the aggregators; or (d) a combination of these.

An MLS's consumer-facing search engine DOES NOT have the same data source as broker IDX sites. An MLS's consumer-facing search engine is NOT an IDX site. (If you doubt me, look up the definition of IDX in your rules.) A broker that takes part in IDX could still opt out of having her listings appear on the MLS's consumer site (and vice versa). This is an important distinction, but it is frequently misunderstood.

Google is not interested in indexing only active listings. Its expressed intent is to index everything! It really cannot wait for folks to submit or upload their content. Instead, it sends out an army of 'spiders,' 'robots,' and 'web crawlers' to gather web pages for indexing. Of course, those animistic terms are just metaphors – there are no robots or spiders. A web crawler is really a series of software applications designed to read web pages, follow the links on them to other web pages, and continue in that way, until they have visited and gathered all the content on the web.

Google's web crawlers go a step further. They gather complete copies of most of the web pages that they view, copies that Google retains on its servers to facilitate indexing and searching. These are called 'cache' copies. The cache copies are really what you are searching when you use Google, and it is based on the cache copies that Google creates indexes.

NEXT: Indexing on search engines

-Brian

Search engines indexing IDX sites Part I: What's the beef?

Just under two months ago, the proverbial stuff hit the fan when a story surfaced that the Metropolitan Indianapolis Board of REALTORS® (MIBOR) told a broker to prevent Google from indexing her IDX web site, citing a common IDX rule:

Participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent "scraping" or other unauthorized accessing, reproduction, or use of the MLS database. (Section 18.2.2 of NAR's model rules, NAR Handbook on Multiple Listing Policy, 2009 ed.)

MIBOR had confirmed its interpretation by contacting NAR's technology and policy staff: In their view, what Google does is technologically indistinguishable from 'scraping.'

What followed was a flurry of blog posts and comments. Some criticized MIBOR's decision as failing to understand Internet marketing and the importance search engines play in real estate search, accused MIBOR and NAR of seeking to favor Realtor.com and MLS public-facing web sites at the expense of broker-owned sites, and accused NAR and MIBOR of not understanding the technology (good example is Rowles). Most criticized the MLS and NAR, a few took more neutral tones (e.g., Matt Cohen, Mike Wurzer), and hardly anyone dared to defend the associations (but see Victor Lund and an anonymous agent on R.O.B.).

NAR's multiple listing policy committee met in May in Washington DC and considered a hastily prepared clarification to the rules that would have changed 18.2.2 to read:

Participants must protect IDX information from unauthorized uses. This requirement does not prohibit indexing of IDX sites by search engines.

The policy committee recommended that the NAR board of directors adopt the proposed change. The board of directors accepted a suggestion from a MIBOR representative to postpone the decision until its next meeting in November. This put NAR in for another round of lambasting.

I don't know whether MIBOR's approach is the right one or not. I do know that there are good reasons for considering that approach, and I plan to explore them in this and the next several posts here.

In summary, I think we will see that allowing a broker to use an IDX site to 'fish' for web indexing results in at least three risks/harms:

1. Listing brokers' expectations about how their peers will use their listings are not met because this use of IDX data is not consistent with the purposes of IDX as they were previously articulated, and the only remedy currently open to listing brokers is opting out of IDX (which means they can't have IDX sites of their own);
   
2. Making an IDX site 'indexable' also makes it easier for nefarious data 'scrapers' to collect data from it; and
   
3. As noted elsewhere, redirecting searches related to certain listings or neighborhoods based on SEO practices rather than broker knowledge does a disservice to consumers.
   

As usual, though, I'll make you sit through a few posts with some background information and to establish some terminology before we get to the payload...

I want to talk first about what a search engine is, discuss how web search engines like Google index the web, and consider search engine audiences and their objectives. Then I'd like to describe the SEO technique I call "IDX index fishing" and discuss why I think the existing MIBOR/NAR view may be justified. Lastly, I'd like to talk about solutions other than those NAR has considered so far.

I'm breaking this into pretty small pieces, because it's easier for me to edit that way, and I've frequently been reminded that most of y'all do not want to read five-page blog posts. Along the way, I encourage comments and private emails to correct me, argue with me, etc. You know the routine.

-Brian

“Brother, can you spare a disparaging remark?”

Blog posts and commentary on broker web sites

Just read an article on Realty Times that touches on the issue of blog posts and commentary on web sites hosted by brokers. A similar issue is making the rounds on the NAR legal counsel discussion group. Victor Lund of WAVgroup also prepared a presentation that I pinch-presented for him at the attorney meeting at the MLS Connections conference in Portland in April; it addressed this and some other 'Web 2.0' legal issues facing the industry.

Based on what I've read, I think we have at least four separate issues:

One is where a blogging broker makes negative remarks about another broker. We would handle these matters under Article 15 of the REALTOR® Code of Ethics, if the statements are false or misleading. Examples:

1. "Joe Broker tried to screw my clients by holding onto earnest money that should have been returned."
   
2. "Joe Broker is very difficult to work with because he is too aggressive in negotiations."
   

Two is where a consumer on one broker's site makes negative remarks about another broker. Article 15 still applies, and the broker operating the site would have an obligation to take down the negative remarks if they are false or misleading. (I'm not sure what the process would be for learning that.) Examples:

3. "Joe Broker tried to screw us by holding onto earnest money that should have been returned."

4. Same as example 2, but comment made by consumer.

Three is where a blogging broker makes negative remarks about another broker's listing. This seems like an issue under Article 12 of the CoE, provided the statements are not "honest and truthful." Example:

5. "I showed 123 Elm last week, and I'll advise clients not to bother in the future: unwashed dishes in the sink, basement smelling of cat urine, shower covered with mildew."

Four is where a consumer on one broker's site makes negative remarks about another broker's listing. Not sure whether the Code addresses this directly (is there a standard under Article 12 corresponding to 15-2?) Example:

6. "My broker showed me 123 Elm last week, and it was terrible: unwashed dishes in the sink, basement smelling of cat urine, shower covered with mildew."

It's also possible to have a hybrid that appears both to criticize the listing broker and the listing, and such a comment could be made either by a broker on her blog or a consumer visiting another broker's site. Example:

7. "Considering the sorry state of this property, it was unprofessional for listing broker to advise a list price of $300k."

Note that the seller's opt-out of 'third-party commentary' on VOWs (Section II.5.c of the VOW Policy) affects only example 6 and (depending on who made the comment) example 7. The VOW Policy provides that a "Participant's VOW may communicate the Participant's professional judgment concerning any listing" as long as it is not factually false. In other words, a broker displaying another broker's listing can criticize the listing (honestly and truthfully) on her own VOW or web site, without violating the Code or the MLS rules, I think, even if the seller has opted out of third-party commentary on VOWs. (Based on this, I read "third-party commentary" to mean someone other than listing broker or seller (the "first party"?) and other than displaying broker (the "second party"?).)

I'm curious if folks would be willing to offer some examples of actual posts that we can look at and discuss. To do so, send me an email with the address of the "offending" post; I'd prefer not to point at specific web pages here on MLSTesseract, because (1) I'm not interested in bringing the 'law' down on anyone's head and (2) comments often get deleted or edited on the source pages, and I don't want to refer to a "problem" on a site that's already corrected it.

-Brian