The risks of Web 2.0

There is always a downside.

I like to talk about ecosystems, open APIs and how collaboration in technology can yield big benefits for the nonprofit sector.

del.icio.us, the social networking site recently acquired by Yahoo, went down this weekend. Lots and lots of website that depend on the del.icio.us API can't get access to data and are also down. Folks who display del.icio.us links on their web pages instead display a "we are down" message. Even companies like Flock, which to a certain extent build their business around del.icio.us, feel the pain with large parts of this product's functionality down.

The ecosystem of open APIs is just as susceptible to disaster as a natural ecosystem. The thing to remember is that an ecosystem is often not as fragile as we might think.

Both Flock and CiviCRM use a very simple strategy to address dependence on resources we have no control over: redundancy. In Flock, they have support for other social bookmarking tools. In CiviCRM, our dependency is on map providers like Google. We build in redundancy by supporting Yahoo maps.

The issue is not that dependency is bad. Leveraging other peoples' investment can make your offering more powerful. How you deal with dependency, how flexible your systems are. and whether you have redundency are all factors that make a Web 2.0 technology like CiviCRM better or worse.

technorati tags: del.icio.us, web20, net2, civicrm

Google and AOL: Flexible definition of evil?

[updated 12/17]

From the NY Times:

Google, which prides itself on the purity of its search results, agreed to give favored placement to content from AOL throughout its site, something it has never done before.

From the Wall Street Journal:

As part of the deal, AOL would be able to sell advertising among the search results provided by Google on AOL Web properties. AOL's sales staff would also sell display ads across Google's network of Web publishers.

From the Washington Post:

Google's search results, based on equations that rank them according to relevancy, will not be changed as a result of the new partnership with AOL, sources said.

Clearly somebody got it wrong. I like John Battelle's idea of a trial balloon.

So, preferential ranking of AOL content in Google search results. Google seems to be careening toword a standard corporate path of profits over people/values. For those who watched the film The Corporation, you might remember they made the arguement that corporations could be clinically diagnosed as psychopaths. Psychopaths might have a different definition of evil than you or I.

As much as I love mash-ups and open APIs, I think it is time for the nonprofit technology sector to seriously look at providing their own technology services. Anyone want to contribute a little open source coding to write an integration between Dataplace and CiviCRM for a non-corporate mapping solution?

Social Source: Web 2.0 Collaboration

[repost from August]

I wrote this in reaction to some conversation that has been happening regarding NTAP collaboration.

NTAP Collaboration in a Web 2.0 World

Collaboration among nonprofits is a well-studied subject with fairly well defined best practices. The modern technological, Internet-enabled “Web 2.0” world changes the logic of collaboration.

In community development, collaboration can best be described as a process in which personal relationships are built, trust is established, and collaboration begins. In a Web 2.0 world, collaboration and trust is the starting point rather than the ending point.

Traditional Collaboration

One of the key best practices in collaboration is to recognize that the relationship between two organizations goes through a natural evolution that is anchored in trust, communication and personal relationships.

The initial stage is a competitive relationship between organizations, competing for funding, publicity or other resources, generally perceiving the relationship as a zero-sum game.

As organizations start forming personal relationships, getting to know one another, they can begin to cooperate…neither organization changing their actions or plans, but getting together when their interests coincide.

Coordination begins the process changing organizational behavior to accommodate ones partner and begin achieving more together than could be achieved individually, even through cooperation. Enough trust has been established that a phone call about a coordination opportunity is warmly received and seriously considered.

Finally, once communication and trust are strong, organizations can plan together, coordinate their activities, and form the basis for long term collaboration.

Web 2.0 Collaboration

In a Web 2.0 world, collaboration is the default world view.

Technology is often critiqued for depersonalizing human activities. In Web 2.0 collaboration, the depersonalization of the process of collaboration generates unique opportunities and significant benefits.

Web 2.0 Collaboration begins with a single organization offering collaborative opportunities to anyone off the street. The default posture is trust of potential collaborative partners, rather than the expectation that trust is built over time through personal relationships.

The Google Maps API is a good example. Google offers any individual or organization the ability to collaborate by creating a mashup of their data and a Google map. People have displayed everything from Chicago crime data to Craigslist listings of apartments for rent via the Google maps interface.

Google offers the collaboration opportunity openly, trusting collaborative partners first, before creating personal relationships. If people misuse the API, the personal relationship kicks in that allows Google to pull out of the collaboration.

The characteristics of collaboration in a Web 2.0 world include:

• On-demand. Any organization can have on-demand collaboration with a partner.
• Highly scalable. Traditional collaboration is bounded by the limits of human-to-human relationships, machine-to-machine relationships are much more scalable.
• Independent of personal relationships. Since technology (APIs) rather than humans (phones) handle the collaboration, one-to-many and many-to-many collaborations can form and grow at a much higher rate.

Web 2.0 Collaboration Opportunities

The primary opportunity is for funders to support the “opening” of organizations’ proprietary outlook and systems.

Compumentor and VolunteerMatch have CRM systems with 50,000 and 30,000 nonprofits respectively. How can they open up their proprietary systems to provide a Web 2.0 collaboration opportunity?

Perhaps a partnership with Google or Yahoo enables adwords-like functionality on their websites. Compumentor and VolunteerMatch might derive revenue from such an arrangement, or require reciprocal arrangements with organizations, driving traffic into software sales or volunteer opportunities, respectively.

Technology producers like Compumentor, NTEN and Social Source Foundation can be supported in producing open technologies. Funders making key contributions… bringing a tool like Techfinder to a website run by an organization that neither NTEN nor Compumentor have relationships with… can jump start the sharing of technology.

CRM software built by the Social Source Foundation can be integrated with VolunteerMatch’s volunteer systems, providing seamless volunteer management or with Network for Good’s donation services, to provide seamless donation management.

All of these collaboration opportunities revolve around internal decisions at participating organizations to “open up” and put technology in place that facilitates and helps to manage openness.

Culture: The Only Significant Barrier

Most of the major technology nonprofits emerged during the dot com boom and are locked into proprietary “portal” business models that companies like Yahoo have long since abandoned.

Moving to a culture where the first question is “how can others leverage what I’m doing” rather than “how can I protect myself from other leveraging what I am doing” is a prerequisite for collaboration in a Web 2.0 world.

Take 15 Minutes to Help Nonprofit Technology

Compumentor has put together the NetSquared project which is an interactive online effort to

• share stories about how web-based technologies are impacting people’s lives
• build toolkits for nonprofits around the globe to help extend their good work

They need some more content. Contribute a case study of a sucessfull example of how nonprofits have used emerging technologies (eCRM, web, email, wikis). Case studies can be contributed on their website (you'll need to log in first).

More interesting to me, they are asking folks to answer four questions about nonprofit technology. You can submit your answers on the NetSquared Website.

• What's *really* new on the Web, as opposed to buzzwords and soundbites?
• Which tools best embody the new opportunities from your point of view and why?
• Who's doing the best work with the new tools (technically or in terms of social benefit or both)?
• What's the bad news? What are the greatest barriers preventing web-based technology from producing social change?

Very interesting to see the different views of the nonprofit, nonprofit technology, and technology communities.

When Technology Decisions aren't Technology Decisions

The guys over at Greenpeace have launched a big advocacy project called Melt. They have decided to "roll their own" technology platform for the effort. In the comments, I think there is a more compelling case for what why they rolled their own... the price of what appears to be a very high quality development shop (Thoughtworks) was the same/ less than proposals for folks using technologies like Drupal/CivicSpace.

In nonprofit land we have been making the case for years that technology decisions are "neutral." You develop your requirements, get vendors to bid on the project, and make a decision about what best matches your requirements. And over the past 10 years the sector has invested millions of dollars of donor money in building the same systems over and over again.

Now that open source has entered the conciousness, projects are being created under open source licenses, but the basic technology decision is still: (1) How much? (2) Quality of the people delivering and (3) Are the requirements met?

Around CiviCRM we are trying to meet these three questions head-on for customers. We are recruiting high quality shops that can deliver low cost solutions that meet customer requirements.

But we what we really care about, is leveraging the kinds of investments that Greenpeace is making into software that benefits the entire sector. Greenpeace is going to spend hundreds of thousands of dollars. By participating in existing ecosystems, that investment could be leveraged to help the broader environmental movement, disaster relief, and small grassroots groups accross the world.

Greenpeace made a rational choice for them. They might even have plans for spending the million dollars or so it would take to create a sustainable community of technology firms, customers, and users around Melt.

But it is expensive and risky to go down this path. When ecosystems of nonprofits (CivicSpace Labs, the Social Source Foundation), firms (twenty plus consulting firms that operate in the ecosystem), and users (500+ installed sites) already exist, we hope major players like Greenpeace would also factor in how their investment could impact the "public good."

[P.S. Greenpeace has had lots of challenges with vendors who have made similar arguements in the past so I undertand their desire to work with a high caliber vendor. I would simply hope other major efforts in the future factor in the opportunity to leverage their technology investments to the benefit of the broader nonprofit sector.]

"Betting against FLOSS is like betting against gravity"

Gotta love Sun. Jonathan Schwartz on open source Solaris:

Betting against FOSS is like betting against gravity. And free software doesn't mean no revenue, it means no barriers to revenue. Just ask your carrier.

He draws the comparison between Cellular companies giving away free handsets (and the customer busing the service and support) and companies giving away software (and the customer buying the service and support).

Now how do we start explaining this to the nonprofit community? There is still a vendor involved, we're just reducing the cost of technology by creating licensing software under an open source license. http://www.openngo.org/

Katrina PeopleFinder Project Metrics Part II

The PeopleFinder project mobilized over 3,000 volunteers to accomplish these goals:

1. Enter unstructured data on refugees from forums across the web to the highest data quality standards possible with volunteers giving a little as one hour of their time.
2. Enter data from databases across the web into the central database via the PeopleFinder Interchange Format
3. Minimize duplicate records
4. Support other organizations in implementing the PeopleFinder Interchange Format
5. Make the central database avaliable to be searched
6. Use the Salesforce API to implement innovative technology solutions to the missing persons problem

What we accomplished:

The project started on September 1, 2005 with the Social Source Foundation, CivicSpace Labs and the Salesforce.com Foundation kicking off the community.
By September 5, we had finalized the Peoplefinder Interchange Format (PFIF), a technical standard for storing and exchanging refugee data.
By September 6, virtually every message board post was hand-entered by volunteers into the PeopleFinder database (100,000 records).
By September 10, almost every missing and found person record on the web was searchable at www.katrinalist.net (350,000 records).
By September 19, over 620,000 records are searchable.
In its first two weeks, the site processed over 500,000 searches
By October 2, the site had processed over one million searches and 649,015 records were searchable.

Even a month after the disaster, we received anecdotal stories of our impact:

> Dear David,
>
> I live in Burbank, ca, got home from work tonight (6pm PST)and had
> a phone messge from a friend here in LA. She lives on skid row but
> was born and raised in New Orleans.
>
> Her elderly mother and son and sister were in the katrina affected
> area, and she had tried to find them via phone calls and the
> internet, but could not. The desperation was thick in her voice
> message, she said she was very worried.
>
> I plugged in my WAN, and went to work. In 5 minutes, i kid you
> not, 5 DAMN MINUTES, i found her son, with a contact email and
> phone number. I set her up a hotmail email account, sent an email
> to him for her, then called her with the phone number. He is in
> Jacksonville NC.
>
> She said it was "only" midnight in Jacksonville and she is going to
> try and call !! I just hung up and her entire tone was happy and
> excited!
>
> I Wanted to send my deepest thanks to you and your crew (all 3000+)
> for giving me the tools to help my friend. You guys should run
> for office.
>
> Anyway, i cant thank you enough for all your hard work and
> sleepless nights. Acts of compassion and generosity like this give
> me hope for the human race. May God rest his rising star on all of
> you and bless you all the rest of your days on this earth.
>
> Peace,
> Sandy

Katrina PeopleFinder Project Makes Discover Magazine

A gentleman, Dion Hinchcliffe, put together a very nice graphic of the Katrina PeopleFinder Project and an interesting take on a Discover Magazine article on our efforts.

Technology is a tool, not an answer.

CiviCRM serves four very big communities, the nonprofit, nongovernmental organization, advocacy, and political spaces. I love the advocacy & political guys because they are way ahead in their thinking. Aldon Hynes offered a great thought today:

Too many of the neo-techno-utopians fall into the same old thinking that technology is the panacea. Really, I suspect that in politics, as with so many of the social issues the world faces, the solution is getting more people to connect with one another, to share their thoughts, hopes, dreams and ideals. To the extent that technology helps get people to connect it can help address social issues. To the extent that it is even perceived as preventing real social connections, it is part of the problem.

In nonprofit technology, especially constitutent relationship management (fundrasing, case management, advocacy, etc.), technology needs to be evaluated to the extent it connects human beings to be more effective.

When we were thinking through our marketing pitch for the Social Source Foundation, we really struggled with efficiency vs. effectiveness. Human relationships lead to effectiveness. Efficiency is the realm of widgits and 4% reductions in operating expenses.

If the technology is really about the people, perhaps the language and message needs to reflect that.

CiviCRM serves three very big communities, the nonprofit, nongovernmental organization, advocacy, and political spaces. I love the advocacy & political guys because they are way ahead in their thinking.

Too many of theneo-techno-utopians fall into the same old thinking that technology in their thinking.

Announcing CiviCRM 1.2

The development teams keep putting out pretty amazing stuff. Probably the best feature in CiviCRM is you can define custom fields and have registered web site users maintain their own information about themselves (CiviCRM Profiles). But it does soo much more...


We are pleased to announce the latest release of CiviCRM, version 1.2! CiviCRM is a web-based, open source, internationalized, constituent relationship management (CRM) application, designed specifically to meet the needs of advocacy, non-profit and non-governmental organizations. As an open source solution, any company, organization or individual can download it, adapt it, modify it and use it without paying license fees.

CiviCRM is currently intended for qualified consultants who are considering deployment of a CRM solution for small to medium-sized non-profits. Organizations considering CiviCRM to meet specific needs should carefully evaluate CiviCRM.

Some Key Benefits of Deploying CiviCRM

• Unified view of every constituent. Store information about individuals, organizations and households and your interactions with them.
  
• Designed for Advocacy/NPO/NGOs. CiviCRM was designed for organizations that engage in advocacy, community and political organizing, and non-profit work.
  
• Seamless integration with web sites. CiviCRM integrates directly into popular open source CMS packages, including Drupal and Mambo. Registration and visitor interactions are logged directly into the system, including end-user maintenance of their own addresses and custom fields.
  
• Internationalized & Localized. CiviCRM was built from the ground up as a product intended to be used globally. It can store data in many localized formats and supports most languages globally.
  
• Open source & open standards. Licensed under the AGPL, users can make any modification to the software, can benefit from modifications made by others and can effectively interoperate with other applications.
  
• Affordable and cost effective. CiviCRM is available without a license fee and is supported by a community of nonprofit-focused consultants.

What’s new with 1.2?

• Improved Data Import/Export. Import and export virtually any CiviCRM data field or set with an easy-to-use interface.
  
• Improved “Profile” handling. We’ve optimized the display performance of our search. You can now search contacts quickly and efficiently. You can also now search on and display any CiviCRM field.
  
• Improved API. We’ve implemented additional API calls including search, group membership, custom fields and more. Remote access via SOAP is now supported.
  
• CiviMail. High volume mass emailing capabilities with the ability to track click-thrus and open rates. (Developer Release)

Demo, Downloads and Documentation

• Test drive CiviCRM 1.2 on our demo site
  
• Download CiviCRM here
  
• Installation instructions can be found here
  
• Documentation, including a Quick Start Guide and a detailed Administrator Guide can be found here
  

CiviCRM in a Nutshell

CiviCRM is a web-based CRM application that can be downloaded and installed either locally on a server, or in a hosted environment. It can be used as a powerful contact database application that allows you to record and manage information about your various constituents including volunteers, activists, donors, employees, clients, vendors, etc. Keep track of conversations, events or any type of correspondence with each constituent and store it all in one, easily accessible and manageable source.

Technical Requirements:

Runs on any platform that supports:

• PHP 4.3+
  
• MySQL 4.0 or 4.1
  
• Drupal 4.6.3+ or Joomla 1.0.3+/Mambo4.5.3+
  
• Joomla/Mambo version of CiviCRM only supports PHP4 at this time

Key Features in Detail:

• Segmentation Tools. Use groups, simple and searchable tags, and/or relationships to segment constituents.
  
• Extensive Configurability. CiviCRM is highly configurable, allowing you, in most cases, to configure it to work with your existing business processes. Unlimited locations, addresses, phone numbers, emails and custom data fields allow most unique needs to be met.
  
• Internationalization & Localization. CiviCRM can store CRM data in many localized formats and supports most languages, currently including Brazilian Portuguese, German, Polish and Spanish.
  
• Contacts. Store common nonprofit contact data (individuals, organizations, and households) that support donor management, case management, voter, and advocacy applications.
  
• Relationships. Understand the relationships between any two contacts with standard (volunteer, employer, head of household) relationships or create your own unique custom relationship types.
  
• Activities. Record standard activities (phone call, meeting, email) for any contact or create your own custom activities that meet your needs. External software can use the CiviCRM Application Programming Interface (API) to register activities with any contact, providing a comprehensive central repository of CRM information.
  
• Smart Groups. Create smart groups based on any search criteria or create standard groups that are simply lists of contacts. The membership of a smart group changes automatically according to that moment’s search results.
  
• Custom Data. Create unlimited custom data fields in virtually any format, including radio buttons, drop-down menus, etc. All custom fields are searchable and can define a smart group.
  
• Support Multi-site Organizations and Networks. Centrally store data across multiple organizations or web sites.
  
• Import and Export Functionality with De-duping. Import functionality intelligently maps CiviCRM fields to imported data and checks for duplicates based on user-defined criteria.
  
• Robust Permissions. With Drupal integration, access to certain groups of contacts can be limited to specific users, offering a way for volunteers to manage small portions of a larger CiviCRM database.
  
• Website Integration. CiviCRM is integrated with both Drupal and Joomla/Mambo. Web site registrations automatically become CiviCRM records and individuals can maintain their own CRM record. Offerings like CivicSpace 0.8.2 integrate e-mail blasts, event, volunteer and petition functionality with CiviCRM.
  
• Application Programming Interface (API). A fully documented API exposing all major functionality of CiviCRM. For instance, you can search the database, register interactions with constituents, create or update contact information, etc. This allows CiviCRM to realistically be a central repository of virtually any nonprofit’s CRM information.
  

Future Releases

CiviCRM is in active development and is constantly improving. Some key features to look for are:

• CiviMail is available in 1.2 as a developer release. In future releases, we will update this high-capacity email broadcast tool with more user-friendly access.
  
• CiviCRM 1.3 will include CiviDonate, a donor management module and online donations solution.

About the Social Source Foundation

The Social Source Foundation is a 501-c-3 nonprofit creating internationalized, open source software of uncompromising quality for the nonprofit and nongovernmental sectors. Social Source Foundation is one of many partners in the creation of the CiviCRM platform, providing primary engineering support for the software.

How Do I Participate in the CiviCRM Project?

Interested parties are encouraged to participate in the development of CiviCRM. This can take the form of providing use cases, feedback on existing functionality, feature suggestions, code contributions, documentation contributions and beta testing. More info...

Open Source & ICT & Values

I am a member of the bytesforall email list, a fairly academic discussion of ICT for developing countries. There was a recent debate about open source vs. Proprietary software that offered a couple interesting thoughts.

First, I've never seen a moderator like Frederick Noronha, who wrote a fantastic synthesis of the debate. Makes me happy to be on the list.

Second, he had nice things to say about me. :)

Most importantly, Richard Stallman has a quote I think we need to think seriously about in the nonprofit sector.

"The choice between free (freedom-respecting) and proprietary
(user-subjugating) software is not a technical choice. It is
an ethical and political issue about people's freedom. To be
neutral on issues that merely concern technology is fine. To
be neutral on ethical and political issues about freedom is
nothing to be proud of."

In the commercial world, software leads to efficiency, which leads to profits. Profits are kind of like software (IMHO)... they have no moral or ethical basis. It is neither moral or immoral to earn a profit.

In the nonprofit world, software leads to efficiency, which leads to organizations capable of doing more good. Doing good is in and of itself a moral and ethical issue.

When we look at open source in the nonprofit sector, I think it is important to at least acknowledge the values issue... why would I support proprietary software solutions that limit the number of nonprofits that can increase their efficiency and do good?

NetSquared: Web 2.0 Collaboration in Nonprofit Technology

Love Compumentor, their people and the whole basic idea behind NetSquared (http://www.netsquared.org). Netsquared is basically a Web 2.0 nonprofit technology community process that leads up to a conference/ gala event that will happen 196 days from now.

I would like to pose a challenge: conferences and "Gala events" are to often about thinking rather than doing. I challenge the folks in this community to spend the next 196 days doing. And spend the conference and gala celebrating what has been done.

I wrote a paper a bit ago talking about Web 2.0 Collaboration for Nonprofits. The basic thesis is that Web 2.0 Collaboration is about "Moving to a culture where the first question is “how can others leverage what I’m doing” rather than “how can I protect myself from other leveraging what I am doing.”

At the Social Source Foundation, we built the CiviCRM software to provide nonprofit-specific constitutent relationship management functionality. We answered the question "how can others leverage what we're doing", by integrating it with Drupal/CivicSpace, the software that runs the NetSquared website. We also published an open API that allows people to develop new software using the core CRM functionality of CiviCRM.

We would like to spend the next 196 days working with folks to deliver some "doing." What might be good doing? We might work with volunteer match to use APIs to integrate their wonderful volunteer recruitment system with CiviCRM, allowing a nonprofit to "automagically" store CRM information about VolunteerMatch volunteers.

We're already working with LINC (Low Income Networking and Communications Project of the Welfare Law Center) to build a robust, free and open source CRM solution specifically designed for grassroots organizing groups.

We'd like to work with you on radically improving constitutent relationship mangement in the nonprofit sector. We've taken the first step by publishing our documentation, specifications, APIs and code under an open source license. Now we'd like to actively help others leverage what we're doing. Visit http://www.openngo.org/ for more information and drop me a line if you'd like to join us in celebrating Web 2.0 achievements in 196 days.

Katrina PeopleFinder Project Metrics

For a loose coalition of 3,000 volunteers, we seem to be doing pretty well.

As of October 2, 2005, the www.katrinalist.net database has 649,015 missing and found persons records. Over 1 million searches have been performed since it went live on 9/6/05, including 16,702 searches since Monday, 9/26/05.

We have a number of anecdotal stories of matches through the katrinalist.net website and are currently working to make sure the PeopleFinder approach and infrastructure are avaliable for the next disaster.

ShelterFinder needs Volunteers

After the immediate aftermath of the hurricanes, there is still a need for clearer information flows to and from shelters.

http://katrinahelp.info/wiki/index.php/ShelterFinder

Longer-term evacuee shelters will need longer-term help and support, but its still not easy to get a clear overview of

• where shelters are
• current shelter status
• how to contact shelters
• how local volunteers are already helping
• how new volunteers can also help

We need your help. Please send me an email at dgeilhufe@socialsourcefoundation.org and I will connect you with the right folks.

House is fine and we'll be back in Beaumont in a few weeks

So hurricane Rita basically went right over my house.

We got back to see out place last week and there was no significant damage, so that was a relief. Our neighbor, though, had a huge tree fall in their living room.

Once they get the electricity back on in Beaumont, we'll head back and start the clean up and repairs.

Personal history of the Katrina PeopleFinder Project PART I

A personal history of the Katrina PeopleFinder Project PART I

The term “social source” is something I have used for half a decade to describe collision of nonprofit technology and the open source movement. It tries to capture the idea that technology can be harnessed for a social mission by employing community development, online community, and web 2.0 strategies.

The PeopleFinder project started for me with a fairly simple social mission. Some folks wanted to put up a website that included CiviCRM, the open source nonprofit constituent relationship management system being developed by a bunch of us at the Social Source Foundation.

Being a good cause and a good test of our technology, we agreed to help install CiviCRM in www.neworleansnetwork.org. Its use would be to power a “peoplefinder” feature, like so many others on the web, to help connect evacuees with one another.

And then I thought about it for a second.
(1) Why build yet another small scale solution to a large-scale problem.
(2) We built CiviCRM to solve major, large-scale nonprofit effectiveness issues related to constituent relationship management.
(3) Open and distributed systems can scale to provide real solutions to national problems.

As this stuff was formulating in my head, I drew up the initial fields for the peoplefinder thinking that we could aggregate all the evacuee sites on the web. Never once did I think there could be a single, “master” database of evacuees. Instead I thought about ways all the evacuee sites could “talk” to one another.

So we needed a data standard with the right fields so that all these bulletin boards and online databases could interoperate. I’m not sure at what point I decided this was going to be a national solution to the problem rather than a small community based web site, but on September 1st, I observed in an email, “Seems like we could bang something usable out in a couple days, get volunteers to do data entry from discussion boards, etc. and have a pretty useful refugee matching solution.”

So then I went out to people I knew and started enlisting help. Andy Carvin, Marty Kearns and Deborah Elizabeth Finn got the first email. Kieran Lal and Zack Rosen from CivicSpace Labs were already involved and they brought in Steve Wright from the Salesforce Foundation.

And this kind of became the ethos of the PeopleFinder project. Send an email out about what needs to get done. People respond to that email and take charge of getting things done. Magically, a solution appears and you’re not quite sure what exactly happened, but you’re trilled that there is now a solution. And you move on to the next thing.

By the 2nd we had a comprehensive list of missing persons sites tagged in http://del.icio.us/tag/peoplefinder. If we wanted to aggregate the bulletin boards and databases, we would need a dynamic, living and scalable list. Del.icio.us did the trick. Note the first use of an open technology…didn’t have to buy it, could just use it to do good. This is an important theme, the technology has to be pre-positioned, accessible, and you can’t need to “ask permission” or even involve the folks that “own”/maintain the technology to use it for your purposes.

About this time, Salesforce.com Foundation committed to providing the back end database and search engine. My motives in engaging Salesforce were twofold. First, they are good guys, committed to open standards…if only they were open source ☺ Second, I felt it important to get a big corporate player involved in the hopes that they could move resources latter on in the process, though their technology is pretty cool too. (Gotta remember, I’m part of the team building the open source nonprofit CRM—I think the nonprofit sector needs a Salesforce.com class solution that meets their needs and is open source :).

The Salesforce.com Foundation and the folks in the company have good hearts, do good work, and we were blessed to have them on the team.

Also on the 2nd we put up a mailing list, katrinadev, because it’s the Internet and you can’t do a project without a mailing list, and recruited folks and did countless other tasks.

On the 2nd we also made a critical technology decision… use a distributed technology like RSS to solve the problem of 20 different evacuee databases. Rather than force everyone to go to a central database, lets make EVERY database central by syndicating evacuee data. At this point the Godfather of the PeopleFinder Interchange Format (PFIF), Ka-Ping Yee, rode in on his white horse. I’m pretty sure Zack Rosen roped him into this, but I don’t actually know for sure how Ping got involved.

Actually, I thought the spec took 24 hours longer to write than it should have, but I have no technology skills, and, as it turned out, should have just trusted Ping and Jon Plax to do a good job… cause they did a stellar one.

I’m not even sure what we actually did the 2nd… Andy Carvin was great at helping flesh out the idea and introducing us to bloggers and others that could spread the call for volunteers. Other folks “spread the meme” and people kept popping out of the woodwork to do stuff.

I think it was the 2nd that Ethan Zuckerman and Jon Lebkowsky were introduced to PeopleFinder—they become critical to the story latter on.

On the 3rd, the Salesforce.com team outlined a project plan and lined up internal resources. Kellan Elliott-McCrea connected us to some guys from Craigslist who were facing the problem of being a repository of missing persons and saw the benefits of a central database (they eventually coded part of the system we used to parse Craigslist into bite sized chunks for data entry volunteers).

We needed a website, a place for a community to self organize. I’ve been a member of the Omidyar Network (http://www.omidyar.net/home/) it started, and have always thought they could be much more than they are. They exist so that more and more people discover their own power to make good things happen. Seemed like a good fit at the time.

On the 3rd Jon Lebkowsky came into the mix from Omidyar (I think). I’ve chatted with Jon a couple times, knew he was a good guy, and basically got out of his way. At some point Jon and Ethan Zuckerman from the Berkman center at Harvard became the point people on data entry. I literally have no idea how the code got written to enable volunteers to do data entry, how the training materials for volunteers got developed, or how that whole side of things happened. I just know there are lots of amazing people that came together and made it work.

Took us about 3 hours to outgrow Omidyar’s interface and move over to http://katrinahelp.info/, a wikipedia site much better suited to the type of self organizing we were doing. Looking back at my email, I think it was Jon Lebkowsky that introduced me to Rudi Cilibrasi, the guy “in charge” of katrinahelp. I just remember trying to connect with Rudi on Skype and having the technology just not work. I ended up calling him (he lives in Europe) and having a 15 minute conversation that just lead us to trust one another… our goals and values were in alignment.

Again Rudi was providing open technology—a wikipedia site. We didn’t need his “permission” to start using the technology (though of course we got it first because that was polite). Even though we didn’t technically need his help to use the technology, he was an amazing resource because he understood deeply how his technology worked and could help others in the community use it to solve problems.

The relationship between PeopleFinder coordinating organizations (Social Source Foundation, CivicSpace Labs, and the Salesforce.com Foundation) and katrinahelp.info is part of what I call Web 2.0 Collaboration. People, technology and organizations whose default position is trust…whose first question is “how can others leverage what I’m doing” rather than “how can I protect myself from other leveraging what I am doing.”
http://socialsource.blogspot.com/2005/08/social-source-web-20-collaboration.html

This whole experience underscores the absurdity of building insular communities that “discover their own power to make good things happen.” Communities exist all over the world and in cyberspace and just need a little infrastructure to catapult them into highly effective entities. That infrastructure of communication and simple directories of what is available needs to be distributed rather than centralized.

Around 3pm on the 3rd we started data entry and started distributing a plea:
http://socialsource.blogspot.com/2005/09/katrina-gave-money-already-give-your.html

At 11:30 pm on the 3rd, I figured it might be good to actually write down an overview of what we had been doing for the past couple days.
http://socialsource.blogspot.com/2005/09/katrina-peoplefinder-project.html

By the 4th, it was pretty clear that we had expanded past the point of being coordinated. So we tried to get some folks to “officially” lead sections of the effort. That effort fell flat on its face mostly because their were people already leading the effort… they were was to busy doing things to have time to list themselves as a leader.

Around this time my role became “human router” I would look at the email stream which was getting absurdly large, and simply connect people with one another. Hey person A, talk to Person B before you do thing C.

The fourth was about details… getting the HTML data entry form from good enough to good, getting the PFIF documentation to a place that it was really useful for developers. Lots and lots of details…

By 3AM on the 5th, we had 10,000 records entered into the database and the volunteer effort was snowballing.

More to come..

Why do we need open source nonprofit CRM infrastructure?

Three distinct disaster response situations come up recently underscoring the need for a solid open source NPO/NGO CRM platform.

1. The Katrina PeopleFinder Project created a single unified database of virtually every missing and found person record on the web (640,000+ records).
2. The ShelterFinder Project created a comprehensive listing of Katrina evacuee shelters (from large Red Cross shelters to small 10 bed churches).
3. The Fluwiki is building an infrastructure to deal with the potential bird flu pandemic.

Each of these projects needs to quickly and efficiently build an application based on individuals and organizations (missing persons, shelters, flue victims). Each started with CiviCRM, but moved on to another solution because CiviCRM isn't quite ready.

The PeopleFinder experience allowed us to optimize CiviCRM, achieving a 100x efficiency improvement. What we need is your support to continue the development of CiviCRM and, most importantly, support its broad adoption in the nonprofit and non governmental sectors.

Open Source means that no one has to ask permission or buy a license to mount a disaster response. We don't have to wait for a philanthropically minded corporation like Yahoo to send 40 engineers to Houston. The nonprofit/ NGO sector can put together a response that leverages volunteer skills into a complete solution within a matter of days (as demonstrated by the PeopleFinder project).

Evacutating for a Hurricane Sucks

I live in Beaumont, TX, and the day after I arrived in Beaumont a few years back, I stayed in town through a "mandatory" evacuation. Didn't do it this time with Hurricane Rita.

Right now, Michelle and I are fine in Dallas in a hotel. The news coverage seems like the house might be fine as well, which would be quite a relief.

Thanks for all the inquiries after our health and well being.

PeopleFinder Project Follow-up/ Thank you.

In early September, I sent out an urgent call for resources for the Katrina PeopleFinder Project. I wanted to follow up, share our achievements, and thank you for your support.

We have combined virtually every missing and found person listing on the web (currently over 640,000 records) into a single searchable database at http://www.katrinalist.net/ in less than a week with an all-volunteer effort.


Goals & Achievements

The Katrina PeopleFinder Project set out to solve a single problem: an evacuee needed to search up to 40 different websites to find out where their loved ones were located and whether they were OK. We laid out three goals:

1. Create a technology specification for easily exchanging evacuee information.
2. Assemble and coordinate volunteers building technology to get all evacuee data into a central database provided by Salesforce.com Foundation.
3. Organizing a massively parallel volunteer data entry project to enter refugee data posted to online bullitin boards into a central database by hand.

We mobilized over 3,000 volunteers and accomplished these goals.

• The project started on September 1, 2005 with the Social Source Foundation, CivicSpace Labs and the Salesforce.com Foundation committing ourselves to the three basic goals.
• By September 5, we had finalized the Peoplefinder Interchange Format (PFIF), a technical standard for storing and exchanging refugee data.
• By September 6, virtually every message board post was hand-entered by volunteers into the PeopleFinder database (~100,000 records).
• By September 10, almost every missing and found person record on the web was searchable at www.katrinalist.net (~350,000 records).
• By September 19, over 620,000 records are searchable.
• Our data is being processed by IBM and the San Diego Supercomputer Center to form part of a central database of evacuees for the Red Cross and Microsoft.

On-going efforts

Two major sister projects have been spawned from the volunteer community:

ShelterFinder is creating a dynamic, comprehensive national listing of shelters (including small community based churches, etc).

PeopleFinder volunteers are coming together to make sure that the technology, procedures and relationships between institutions necessary to duplicate our achievements are ready for the next major disaster.


Learn More

For me, this entire experience is about non-profits having the capacity to leverage technology and massively parallel resources to better fulfill their missions... helping people. The same work I do at the Social Source Foundation.

It is time that we invest in pre-positioning technology, capacity, plans and knowledge so that we’re not building the bridge as we cross the river in the next disaster. I was proud to have led part of this dynamic, distributed effort. If you would like more information, please send me an email and I will connect you to the appropriate person(s).

We are at also offering an overview of the project in a Webinar sponsored by the Nonprofit Technology Enterprise Network (NTEN), HumaniNet, Aspiration, and the Innovation Funders Network. Hurricane Katrina: Innovative Information and Communication Responses is a free online event. You can register at http://www.nten.org/webinars.


Thanks from the many volunteers of the Katrina PeopleFinder Project,
http://katrinahelp.info/wiki/index.php/Katrina_PeopleFinder_Project


And a personal thank you from me,

David Geilhufe
Co-Founder
Social Source Foundation
Connect with me on Linkedin...

537,180 people as of 9/14/2005 5:51 PM

I think we now have most of the Katrina missing and found persons on the web in the database. You can search at http://www.katrinalist.net/

Now the Red Cross needs to get itself together and get a common database together. We keep hearing it will be Katrinasafe.com .... get on with it. There are survivors that need to know where their friends and family are.

We of course, built the PeopleFinder Interchange Format to avoid the need to have a common database, but unless it's widely adopted, it will not solve the problem.

Kieran Lal's Personal Story of PeopleFinder

The PeopleFinder effort has been incredible fast, distributed, disconnected and effective all at the same time.

If you've been involved, I engcourage you to write a personal story of your involvement. I'm writing mine now.

Eventually, we might actually figure out how a bunch of unpaid volunteers created the most comprehensive directory of survivors and missing persons on the web (and A LOT more including ShelterFinder) .

Keiran's story.

Search PeopleFinder Data

Paste this code into your HTML page to provide a search into the Katrina PeopleFinder Project data.

Katrina PeopleFinder Project

Enter a name, phone number, email address, city, zip or neighborhood of the person you are looking for. Powered by the community of volunteers from the Katrina People Finder Project.

346,521 records as of 9/11/2005 10:43 AM

This is our effort in numbers. On the 4th the technology went up. By the 6th, the majority of unstructured data was in the database (craigslist, nola.com, etc.) through thousands of volunteer data entry folks and "scrapes" of Gluf Coast News, IDRC, MSNBC and other structured sources were ready to go into the database, but not there yet.

Unfortunately, it took another couple of days working through technology issues for stuctured data to be bulk loaded into the PeopleFinder database and today we have the comprehensive database we wanted to have up by the 7th.

People ask me where survivors should search. I tell them (1) Yahoo People Finder (good name, I approve) because it constantly is crawling the various boards and resources. (2) Katrina Safe becuase it is destined to be the "official" repository. (3) Katrinalist.net since we have some records that Yahoo doesn't and our data is in a more structured format, allowing potentially better matches with partial information.

We are doing this becuase survivors need to find their loved ones now. Eventually, the technology instrastructure will exist, hopefully based on what we've done here, to make quick massive reaction a plan rather than a struggle that requires Yahoo to fend 40 engineers out to Houston and Microsoft to have engineering teams working around the clock in Houston, or thousands of Katrina PeopleFinder Project volunteers work straight through the holiday weekend.

246,108 Records as of 9/10/2005 5:20 PM

Includes scrapes of Gulf Coast News and IRDC familylinks data. As always our data contains a hyperlink back to the original source so survivors can evaluate the original source of information.

All data is in the PeopleFinder Data Interchange Format and can be syndicated via RSS between organizations (like shelters) and websites.

200,984 records As of 9/10/2005 10:09 AM

Katrinalist.net, the search engine into the Katrina PeopleFinder Project data, has over 200,000 evacuee records in it as of this morning as uploading data from other existing databases has begun. We know there are duplicates in there, but figure folks would rather find 6 records about their brother than no records at all.

The Katrina PeopleFinder Project has another 300,000 records qued up to enter the database.

Katrina PeopleFinder Project: Community-based information

Today was a bit of a roller coaster. PFIF uploads are functional on the server. We parsed our first inbound PFIF feed. Volunteer data entry started up again. We had a little community "crisis of conciousness" as it became aparent that katrinasafe.org would become a defacto standard.

And the community came together again, thought about it for a sec, figured we still had a niche, then redoubled its efforts. Very cool. Check out the Shelter Project, a massively parallel effort to identify shelters and gather PFIF compliant data for the search engine. Current community thinking seems to be that we may be more effective at nationally aggregating shelter data quickly, especially from small community based agencies.

Status of sites implementing PFIF feeds is here.

I have to say that I've been really, really blown away by the response and support that Microsoft has given to this project, both the company and the people. Here in Austin, we've not wanted for anything ... food, drinks, everything is brought in and, in some cases, brought to us at our desks. (Eating fajitas while coding can be somewhat challenging.)

From one of the developers of http://www.katrinasafe.com/, which seems to be on its way to being the central repository of survivor data. We will, of course, be providing date to it.

Next time I develop a survivors database, I'm gonna do it for Microsoft.

First PFIF feed loaded into PeopleFinder Project

I am proud to announce that the first PeopleFinder Interchange Format data feed has been read into the PeopleFinder database. Earthlink, a critical participant in the PeopleFinder Project provided the first feed this morning.

When you go viral and call in the calvalry, sometimes it pays to take a deep breath.

I have issued a call for full time project management in a number of areas to help out with the Katrina PeopleFinder Project. Our all volunteer leadership is pretty worn out, as you might imagine. Lots of new leadership is emerging in the community, so we might yet self-organize our way into an even bigger sucess (email list volumes are pretty staggering, so it may already be happening).

I would prefer a single organization came in to marshal resources, improve communications, help the community set goals and assemble volunteer teams to meet those goals from within the community. Pre-existing relationships go a long way-- you don't have to learn a new person's name every five minutes. It will be chaotic.

The team that will manage the official "selection" (which up to this point has been the first competant person to show up) gave me a great analogy. The first stage of the booster rocket has burnt itself out. We are in that momentary lull between the first booster rocket shutting down and the second booster rocket ignighting. The second stage is all about nourishing and supporting the community of both highly technical and regular people volunteers to self organize. From there, we break free of gravity.

The project management people are human routers, they don't develop project plans and just implement them. They over-communicate and create opportunities for the community to come together over certain goals.

• Priority 1. We need to get very specific offers from organizations. Include resumes.
• Priority 2. The selection team will look at your stuff. Hopefully you have already oriented yourself to what is happening in the community, or better yet, have been part of the community since the weekend.
• Priority 3. The selection team will train and orient the full-time volunteers. This process should take at least six hours, you'll talk to a lot of people, each with a different vision and view on what is going on. It is pretty much that complex.
• Priority 4. The project management team needs to enter the community and just start being human routers, being patient as volunteers check in and out, most not able to keep up with traffice on the email list, or even figure out what changed over the last 5 minutes.

How to help:

1. Contact me. I will forward offers to the selection team.
2. Be patient. Lots of people, lots of activity.
3. Up till now, we have been entriely self-organizing. You are welcome to just wade in. Like our data entry, the process of organization can be viral and massively parallel.

How to collect data on every shelter in America in 72 hours

OK, maybe not 72 hrs, but hey, who thought we could do 60,000 records in a day?

1. Build a database of shelters by calling on volunteers to record the location of every shelter in America (validate the addresses with USPS so you don't get duplicates).
2. Allow volunteers to "claim" shelters. Once a shelter is claimed, they are comitting to putting together an excel spreadsheet with a PFIF record for each person in the Shelter. Probably have to prequalify people and offer them a little training.
3. Volunteer uploads their excel sheet on the web (software can make sure its in the right format).
4. Another volunteer or maybe staff checks to see if the data looks accurate and in the appropriate format.
5. The excel sheet of PFIF records is uploaded into the database.
6. Those results are immediately avaliable at Katrinalist.net and syndicated to any other database on the web.

The prototype for this was posted today in our message boards. Sure wish we had some full-time project managers to implement this stuff so we don't have to waste more time chasing CNN stories so people can understand the value of massively parallel volunteer efforts and I could get some sleep. We could have the software up in 48 hrs, volunteers mobilized in another 24, and the whole process done perhaps 72 hrs latter.

Press discovers Katrina PeopleFinder Project, Now we just need 6 or 8 full time engineering and project managers

I love this Red Herring story. It is exactly as tired as I feel. Key point. Send us engineering teams (actually we more need engineering and project managers).

SF Chronicle
NPR
http://www.alternet.org/story/25177/
http://db.tidbits.com/getbits.acgi?tbart=08235
http://www.worldchanging.com/archives/003437.html
http://www.npr.org/templates/story/story.php?storyId=4834186

Thoughts on Implementing PFIF

The PeopleFinder Interchange Format has strengths and weaknesses. It's strength is that it can syndicate a missing persons request far and wide quickly and easily. It's weakness is that the more personal information you put in the PFIF record, the bigger a threat to privacy it is.

But what if you only put the minimal personal information into your RSS feed? Just the stuff required to identify that this is Bob Smith, lived on Sampson Street, in the 9th Ward, son of Doris and Kathy?

Privacy is not compromised and people who know one another can still find one another. The source URL field of a PFIF record can point back not necessarily to a record containing personal information, but a record containing the name, address and phone of the shelter where that person was located when the record was created. Privacy is safe and you can still find people.

Trusted providers can still exchange large data sets in PFIF with all kinds of personal data in them to facilitate automated matching and notification.

Lets build this stuff, people.

Search over 88,000 volunteer-entered records

www.katrinalist.net is live, allowing users to search 88,000 records hand entered by volunteers from forums accross the web.

Volunteer engineering teams are hard at work preparing additional data from a variety of sources for entry into the database. We hope to provide a robust repository of quality data.

We could use full-time engineering and project management teams. Your small corporation's full-time staff of 4 dedicated to the project for a couple weeks could accelerate our progress and increase our effectiveness. Please consider joining us.

Implement PFIF NOW

Evacuees Find No Easy Way Locate Family Members
by Joseph Shapiro
Morning Edition, September 6, 2005 · From the airport in New Orleans, tens of thousands of medical patients are airlifted to shelters in hospitals in often distant cities. That effort was coordinated by the U.S. military and government agencies. But there was almost no coordination to keep good lists of who was sent where.

The problem can only get worse. The Katrina PeopleFinder Project released the PFIF specification on Saturday to address just this issue of scattered refugees with no good lists of who went where. By Sunday, a massively parallel volunteer effort had hand-entered virtually every missing and found forum posting on the web (almost 80,000 records).

Our volunteers are tired. They worked all weekend and had to go back to their jobs. Companies should contribute an engineering team or two to the cause.

We need engineering teams

The Katrina PeopleFinder Project is an all volunteer effort.

Its crunch time and we need full-time engineering teams. Yahoo got a crawler up becuase they have full time resources dedicated to the project. The Katrina People Finder Project needs full time engineering teams.

Wouldn't you think a Silicon Valley technology company could spare an engineering team for a week to work with us in making refugee information more accessible to the displaced people of the Gulf Coast. Folks need to step up, we have the plan, the vision and the specification. It's just a matter of resources.

We need a salesforce.com administrator, project managers and engineers to implement the PFIF spec for other organizations (http://zesty.ca/pfif/1.1/#life-cycle), and engineers to do scrapes of external websites and put the information in PFIF.

We can create the largest, most useful structured data set on the web. We just need a visionary company to donate a full time engineering team or two.

Overall PeopleFinder Project Goals

The Katrina PeopleFinder Project is a massively parallel volunteer effort to solve the problem of dispirate refugee/ missing persons databases and forums.

1. Enter unstructured data on refugees from forums across the web to the highest data quality standards possible with volunteers giving a little as one hour of their time.
2. Enter data from databases across the web into the central database via the PeopleFinder Interchange Format
3. Minimize duplicate records
4. Support other organizations in implementing the PeopleFinder Interchange Format
5. Make the central database avaliable to be searched
6. Use the Salesforce API to implement innovative technology solutions to the missing persons problem

(1) Is currently implemented by dividing forums like Craigslist into "chunks" of about 25 records through software and/or volunteers. Volunteer data entry people "claim" a "chunk" and enter it into the central database. They follow instructions on how to enter the data to maximize the data quality. This effort entered over 68,000 records in less than 36 hours.

(2) Is currently implemented by software engineers either scraping or transforming existing databases into that PeopleFinder Interchange Format (PFIF) and then loading that data into the central database OR by missing persons database owners implementing the full ProjectFinder Interchange Format (PFIF) wich allows an RSS feed of refugee data to be passed from database to database.

(3) Is handled by trying to coordinate among all the different teams and volunteers so that duplication of effort is minimized. The "chunking" and record claiming process is also critical.

(4) Is handled by contacting organizations to make them aware of the PFIF format and helping them decide whether to implement it. We might also try to provide some volunteer assistance to sites in implementing PFIF.

(5) Is being handled by a search interface into our main Salesforce.com data repository.

(6) Once data is in the repository it should be processed to try to match missing people to found people, facilitate communication and just generally help refugees out in any way possible via data and technology.

This is a massively parallel volunteer effort. Please figure out a small part to play in these very large goals. Enlist some people to help, make sure other people know what you are doing and just go to it.

PeopleFinder Interchange Format (PFIF)

Some amazingly smart folks have completed the data specification that is the cornerstone of the Katrina PeopleFinder Project. Use it. Convince others to use it :)

The PeopleFinder Interchange Format (PFIF) is well described in the FAQ, even with a handy little diagram (http://zesty.ca/pfif/1.1/#life-cycle).

49,995 records As of 9/5/2005 7:52 PM

A volunteer data entry operation of the Katrina PeopleFinder Project has entered information on 49,995 people listed in message boards like Craigslist in about 24 hours. With extraordinary contributions from nonprofits, companies, regular volunteers and key partners like Salesforce.com, we have only just begun.

All our data conforms to the PeopleFinder Interchange Format (PFIF), allowing it to be combined with other structured databases that convert their data to PFIF. Data on missing persons from around the web can be combined into a large, central, searchable repository.

Go here for a link to the PFIF http://192.122.183.218/wiki/index.php/PeopleFinderTech

The next step is to add data to the central repository from structured databases like the Red Cross. They can help by transforming their data into something conforming with the PFIF spec. Their records can then be combined with ours allowing people to search for missing persons listed accross the web. Earthlink is the first organization to implement their database according to the PFIF specification. Great work guys.

The Katrina PeopleFinder Project is a distributed, community effort led by dozens of amazing volunteers and made possible by hundreds of people giving their time. Thank you all.

Why enter data when I can't search it?

We have a massively parallel volunteer effort going on. Step one was to design a data standard, the PeopleFinder Interchange Format (PFIF). Step two was to start up a volunteer effort to enter data in. Step three was to implement the PFIF for data entry of message boards, hence our move from www.neworleansnetwork.org to new servers. Step four is to enter all the data that is in databases on the web. Step five is to make it all searchable.

We are doing steps 4 and 5 right now. A wide variety of volunteers and companies are creating software to bring the existing databases into ours. We actually would prefer them all to implement PFIF so that we can just parse their feed and they only minimally change their site.

The search interface is being lead by Salesforce.com. The status of that project is publically viewable on the project wiki.

We're for real folks. Nothing nefarious going on.... just a lot of tired people on a holiday weekend away from their families for a good cause. Thanks to all of our volunteers. This wouldn't work without you.

Volunteer Data Entry works: 11,643 entries in Salesforce.com Central Repository at 7:13 AM up from about 7,000 at 3 AM

Add this to the 2,600 we did yesterday before we melted down the server and we have put about 14k forum postings into a database format with the help of volunteers. Good job folks!

Another Database? Aren't you part of the problem? NO!

The Red Cross has a missing/found persons database. We have identified another 10+ databases on the web. Web forums like craigslist are posting missing/found persons information there are probably 25 major ones of those.

We are consolodating all this information from all over the web. Hundreds of volunteers are entering craigslist and other forum entries by hand. 10,000 of them since yesterday.

We have published an open data spec, the PeopleFinder Interchange Format (PFIF), that can facilitate all the various databases syndicating information into a single database.

We are an open and community process working as quickly as possible to get information for individuals impacted by the hurricane. Please Join Us. http://socialsource.blogspot.com/

Hard core technology volunteers- hackers, Google engineers, super data heads are critical to the effort. Join the hard core techie email list by sending mail to katrinadev-subscribe@civicspace.org.

The hard core techie Wiki with the data standard is at http://192.122.183.218/wiki/index.php/PeopleFinderTech

3AM Central 9/5/2005. 10,000 records entered. Great Work!

3AM Central 9/5/2005. 10,000 records entered. Great Work!

The Katrina PeopleFinder Project is creating a central repository for all refugee records on the web.

Two main efforts are:
(1) VOLUNTEER
http://192.122.183.218/wiki/index.php/PeopleFinderVolunteer
Massively parallel volunteer data entry of refugee records into a central repository, conforming to the PeopleFinder Interchange Format (PFIF).
(2) TECH
http://192.122.183.218/wiki/index.php/PeopleFinderTech
Volunteer techies effort to publish the PeopleFinder Interchange Format (PFIF) and collect data from refugee databases across the web.

Great work!

http://socialsource.blogspot.com/

http://blog.social-source.com/ got slammed.

We are now at: http://socialsource.blogspot.com/

Thank you.

We need community leaders to step up.

We need community leaders to step up.

Self-organizing is hard. Thank you for investing your time. It will make a HUGE difference.

1. Pick an area.
2. Organize a leadership team on Omidyar Network (links below) [(3) is done already]
3. IM me so that I can publicize who the leaders for each team are and keep this blog post updated with the current information.
4. Leaders should post their contact information so that people can contact them.
5. If you can't lead, just subscribe to the email list (katrinadev-subscribe@civicspacelabs.org) and stand by while we get organized.

Keiran Lal (Kieran AT civicspacelabs org) will organize a conference call and IRC for latter to help with organizing.

To stay informed, send email to katrinadev-subscribe@civicspacelabs.org

If you are involved with the Katrina PeopleFinder Project, or want to be, it is critical that we improve our organization structure as soon as possible. Luckily, Jon Lebowsky has shown us the way. Thanks Jon! You can see how they organized the leadership team by following the Omidyar Network link for the volunteer data entry project.

Self-organizing is hard. Thank you for investing your time. It will make a HUGE difference.


The project has four main goals. Each on needs a few leaders and back-ups.

(1) DATA SPECIFICATION. Create, disseminate and support others in using the PeopleFinder Interchange Format.
Leader1: Ka-Ping Yee (ping AT zesty ca)
Leader2: (kleinpeterj at corp.earthlink.net)
Leader3: ?

(1a) Contact all existing structured data sites ask them to implement the data standard
Leader1: Katrina Data Project (?)
Leader2:

(1b) Help developers implement the data specification
Leader1:
Leader2:

(1c) Whatever else needs to be done.

Organize your leadership team here: http://www.omidyar.net/group/katrinarefugee/news/1/

(2) WRITE SOFTWARE. Write software that gets other people’s data into the PeopleFinder Interchange Format.

Leader1: ?
Leader2: ?
Leader3: ?

(2a) Assign folks to scrape specific databases
Leader1:
Leader2:

(2b) Create Salesforce.com central data repository.
Leader1: Jon Plax (jplax AT salesforce com)
Leader2:

(2c) Other items

Organize your leadership team here: http://www.omidyar.net/group/katrinarefugee/news/2/


(3) VOLUNTEER DATA ENTRY: Organizing a massively parallel volunteer data entry project to enter refugee data posted to online bulletin boards into a central database by hand.
Leader1: Jon Lebowsky (irc.freenode.net #globalvoices)
Leader2: ?
Leader3: ?

(3a) Don’t exactly know since Jon Lebowsky has it under control
Leader1:
Leader2:

(3b) Don’t exactly know since Jon Lebowsky has it under control
Leader1:
Leader2:

Your leadership team is already organized here: http://www.omidyar.net/group/katrinarefugee/news/0/


(4) Market the Katrina PeopleFinder Project and recruit volunteers.
Leader1: Andrew Hoppin (Andrew AT civicspacelabs org)
Leader2: ?
Leader3: ?

(4a) Recruit full, deep leadership teams for all segments of project
Leader1:
Leader2:

(4b) Other Items

Organize your leadership team here:
http://www.omidyar.net/group/katrinarefugee/news/3/

Gave money already? Give an hour of your time right now online. Katrina PeopleFinder Project

Refugees can search 20 web sites for lost relatives and still miss their entry on the 21st web site. There is a need to combine all the refugee data from big databases like Red Cross, large posting forums like Craigslist and many other sources on the web. The Katrina PeopleFinder Project seeks to create a single repository combining as many sources of refugee data as possible from all over the web without interrupting existing momentum.

We need help for both regular people and software engineers. Everybody is critical to building a central repository of ALL the refugee records we can find on the web. The Social Source Foundation, CivicSpace Labs and Salesforce.com Foundation are coordinating hundreds of people and organizations, including Craigslist and Earthlink.

Please consider giving us just an hour of you your time to do volunteer data entry. The PeopleFinder Project is seeking volunteers in four primary areas:

(1) Creating a technology specification for easily exchanging refugee information. A volunteer effort is working to assist online databases in implementing the specification.
Volunteer here (techies): http://www.omidyar.net/group/katrinarefugee/news/1/

(2) Coordinating volunteers that are writing software that takes information from online databases and putting it into a central database provided by Salesforce.com Foundation.
Volunteer here (software engineers): http://www.omidyar.net/group/katrinarefugee/news/2/

(3) Organizing a massively parallel volunteer data entry project to enter refugee data posted to online bullitin boards into a central database by hand.
Volunteer here (regular people): http://www.omidyar.net/group/katrinarefugee/news/0/

(4) Market the Katrina PeopleFinder Project and recruit volunteers.
Volunteer here (marketing folks): http://www.omidyar.net/group/katrinarefugee/news/3/

Katrina PeopleFinder Project

What is the Katrina PeopleFinder Project?
Refugees can search 20 web sites for lost relatives and still miss an entry on the 21st web site. There is a need to combine all the refugee data from big databases like Red Cross and large posting forums like Craigslist. The Katrina PeopleFinder Project seeks to create a single database combining as many sources as possible from all over the web without interrupting existing momentum.

How did the Katrina PeopleFinder Project start?
Community organizers from the League of Pissed off Voters, themselves refugees from New Orleans, needed a tool to help people in the New Orleans and Gulf Coast area stay connected to the communities they love. Enlisting the aid of nonprofit technology assistance providers Radical Designs, Social Source Foundation, and CivicSpace Labs, the site www.neworleansnetwork.org was created with open source technologies designed by and for nonprofits.

Realizing the PeopleFinder tool could be used to aggregate databases and information from around the web, the Salesforce.com Foundation joined the effort, combining corporate technology and resources with the power of the grassroots.

What is the Katrina PeopleFinder Project doing?
(1) Creating a technology specification for easily exchanging refugee information. A volunteer effort is working to assist online databases in implementing the specification.
Volunteer here: http://www.omidyar.net/group/katrinarefugee/news/1/
(2) Coordinating volunteers that are writing software that takes information from online databases and putting it into a central database provided by Salesforce.com Foundation.
Volunteer here: http://www.omidyar.net/group/katrinarefugee/news/2/
(3) Organizing a massively parallel volunteer data entry project to enter refugee data posted to online bullitin boards into a central database by hand.
Volunteer here: http://www.omidyar.net/group/katrinarefugee/news/0/

Who is involved?
The Social Source Foundation, CivicSpace Labs and Salesforce.com Foundation are coordinating hundreds of people and organizations, including Craigslist and Earthlink.

Contact
IM: dgeilhufe yahoo com

Links:
A blog to keep up-to-date (http://blog.social-source.com/)
Social Source Foundation (http://www.openngo.org/)
CivicSpaceLabs (http://www.civicspacelabs.org/)
Salesforce.com Foundation (http://www.salesforce.com/foundation/)
Craigslist (http://www.craigslist.org/)
Earthlink (http://katrina.earthlink.net/)

FINAL REFUGEE DATA EXCHANGE SPEC

Refugees can go to 20 different websites to find information on their loved ones. We are publishing a spec to facilitate data interchange among sites and that would allow the creation of a central database of most refugee databases on the web. We are also connecting database owners with volunteer programmers that can help implementing the spec. Special thanks to Ka-Ping Yee. Peoplefinder is a community effort lead by the Social Source Foundation, CivicSpace Labs and Salesforce.com Foundation.

The "official" data exchange format for refugee data is defined here:
http://zesty.ca/katrina/

If you run a refugee database, please publish your data via RSS in this standard. If you need volunteer programmers to help with implementation IM me at dgeilhufe AT yahoo com.

If you can volunteer to contact refugee databases (Red Cross, Gluf Coast, etc.) and help them implement the standard, please go here:
http://www.omidyar.net/group/katrinarefugee/news/1/

If can be a lead community organizer of programmers actually implementing the standard for websites, or for efforts to scrape databases, please go here:
http://www.omidyar.net/group/katrinarefugee/news/2/

Please diseminate this information far and wide.

Katrina: Gave money already? Give your time right now online.

A significant problem for refugees is that information on people is spread all over the web. If I was looking for a loved one, I could search 20 websites and still miss and entry about them. There are a number of technical solutions people are working on right now, but the old fashioned grassroots approach may be best.

Can you spare one hour today and enter missing persons data from forums accross the web into the peoplefinder database at http://www.neworleansnetwork.org/ ? We have people working on importing the major online databases on the web (Red Cross, Gulf Coast News, etc.) in addition to your data entry.

The first thing we need is community leaders that can help coordinate a massively parallel data entry effort so that thousands of volunteers can enter information from message boards across the web without too much duplication of effort.

The thing we need to resolve first is how to coordinate such a significant effort.

Go to http://www.omidyar.net/group/katrinarefugee/news/0/ where you will be coordinating the volunteer effort.

Please publicize this message far and wide.

Calling community organizers!!

We need some leaders to step in a organize some efforts. Are you up for it?
Email me dgeilhufe --AT socialsourcefoundation --DOT org

Goals we need leaders for:
(1) Organize a massively parallel volunteer data entry project.
(2) Organize programmers to populate a central database from the variety of databases on the web.

Katrina Refugee Database Data Standard

CiviCRM is being used for a refugee database at http://www.neworleansnetwork.org/

So we built a data standard that we are trying to get used in as many places as possible (to facilitate data exchange).

The current as final as we can get it dat standard is here:
http://zesty.ca/katrina/

Hurricane Katrina: Volunteer Data Entry

Information on missing persons is in hundreds of forums accross the web.

If a thousand people spend just one hour today entering information from forums into a database, hurricane victims will have a much easier time finding out the status of friends and loved ones.

We have a repository for this data at http://www.neworleansnetwork.org/ a grassroots website built with grassroots technology (CiviSpace & CiviCRM) by grassroots folks Radical Designs and League of Pissed Off Voters.

Step One:
Figure out how were going to coordinate a massively parallel volunteer data entry effort.
Sites that need to be entered by hand (we can't write software that will put their information in our database) are tagged in delicious.
http://del.icio.us/tag/peoplefinder+unstructured

Step Two:
Write clear instructions on how to participate.

Step Three:
Let as many people know about it as possible.

We are on step one. Send me an email at dgeilhufe at-- sourcialsourcefoundation dot-- org to help with the planning.

Current organizing locations/ how to help with central "distributed" database

OK, things are slowly taking shape.

First email coordination:
Our friends at civicspace labs have put up an email list:

If you are a techie/programmer, please drop me a line at david at-- socialsourcefoundation dot-- org and we'll get you on the mailing list.

Non-techies, please stand by :)

Second IRC:
irc.freenode.net
either #civicrm OR #civicspace
Coordination was going on there last night.

Screen scrapping targets

We want to get as many missing persons records into a single database as possible.

(1) Get techies to build software to "scrape" info from structured sites into a standard datamodel.

(2) Get volunteers to read forums and postings and manually enter that data into a standard datamodel.

Here are some of the places we need to have scraped/ entered into our standard datamodel. We'll of course be syndicating the unified database.

Lists of lists:
http://www.packrat-pro.com/katrina.htm
http://www.blessingsontheweb.com/

Red Cross has got their database up which is huge, so that is another good scrapping opportunity.
http://www.familylinks.icrc.org/katrina/people

The big list so far has been the Gulf Coast News site.
http://wx.gulfcoastnews.com/katrina/status.aspx

Our goal, again, is to combine all these databases into a single data model, offering a unified view of who is OK so refugees don't have to spend their time searching message boards and databases.

Ecosystems are not about design or control

Marnie Webb always triggers a good thought or two:

It’s about turning over part of yourself. In two essays (1, 2) Peter Merholz argues that it’s not about the technology. He writes: “Web 2.0 is primarily interesting from a philosophical standpoint. It’s about relinquishing control, it’s about openness, it’s about trust and authenticity. APIs, Tags, Ajax, mashups, and all that are symptoms, outputs, results of this philosophical bent.”

In a Social Source Ecosystem, openness, trust and authenticity are cultural norms. The actual code and behavior of actors are just "symptoms, outputs and results of this philisophical bent."

A Social Source Ecosystem: Catalysis

Catalysis-- an action or reaction among actors that is triggered by an outside agent--a very small amount of catalytic agent can facilitate a very large scale reaction.

Open source software becomes a catalytic agent that brings new actors into the social source ecosystem. Recently an online petition application was released using the CiviCRM API to store data. An entirely new community of users that care about online petitions will now be exposed to CiviCRM and potentially join our community and leverage our software.

One of the key factors for a successful catalyst is an environment conducive to broad cooperation across organizations, markets, commercial products, and human activities. In the software world, this means standards. TCP/IP, HTML and XML provide the basic technical standards, but for a Social Source Ecosystem, standard representations of actions, online donations, contact records, etc. Become just as important.

In the end, group forming networks are probably the most relevant mechanism for large catalytic impacts in the Social Source Ecosystem.







The implication is that the Social Source Foundation cannot be a central clearing house, but instead focuses on the basic rules and tools that allow any individual or group to immediately use CiviCRM for their purposes, creating a new group around their specific needs.

Social Source Socialist?

Oh, how I love it when corporations make arguements that can be applied in different ways. In the fundraising software space, there is a debate about people filing business process patents for specific online fundraising methods. The most noise comes from software vendors that will lose business if a single vendor is the only one able to offer a specific business process.

One arguement against business process patents is:

Shouldn't technology enable us to do more and to do it more effectively? Restricting our use of fundraising tools limits the number of people we can engage, the volume of donations we will receive, and ultimately, the universe of people we can help.

So I gotta do it:

Shouldn't technology enable nonprofits to do more and to do it more effectively? Restricting nonprofit use of fundraising tools (through expensive proprietary software licenses) limits the number of people nonprofits can engage, the volume of donations nonprofits will receive, and ultimately, the universe of people nonprofits can help.

This is not an arguement to make things free (the social source ecosystem depends on customer revenue), just an arguement to radically reduce barriers to adoption. ;)

A Social Source Ecosystem: Group Selection

In one of my early presentations on the potential of open source software for nonprofits, I had a slide called “Sounds like Socialism.” The slide tried to address a perception that a utopian view of sharing software and innovations simply couldn’t work. In our market-driven capitalist society, if you don't buy it, it can't be valuable.

The concept of group selection injects a decidedly cut-throat capitalism aspect into the concept of Social Source. The Institute for the Future notes,

“…groups work best when their members provide benefits to one another, but many of these prosocial behaviors do no survive through natural selection.” Individuals who effectively compete with other individuals succeed in evolution; those that cooperate are less successful.

How then do we conceive of a Social Source ecosystem? If it’s not one group of individuals and organizations sharing software in a utopian collaboration of nonprofits, what is it?

Group selection is the concept that individuals are not the only ones subject to natural selection—natural selection also operates at the group level—groups of individuals. Cooperation within a group can be a very important asset when competing against other groups.

This has implication for the Social Source ecosystem competing with the commercial software ecosystem, but also for the internal organization of the Social Source ecosystem.

Applying this to the Social Source ecosystem suggests there is no monolithic groups or single leader. Instead, some core principles (open source licenses) simply guide the entire system. Conflict and competition at a wider scale in the Social Source ecosystem will encourage local cooperation in order to compete in the wider group.

In the case of technology like CiviCRM, there is a strong incentive to support the creation of multiple donor database solutions, multiple volunteer management solutions, etc. The competition among those solutions will actually make those solutions stronger.

This leads both to competition among groups sharing volunteer management code for their specific solution, but also supports much broader cooperation as multiple volunteer management solutions share innovations at the CiviCRM level.

Things are getting curiouser and curiouser as I think through how a Social Source ecosystem might work.

Social Source Symbiosis

The Institute for the Future's second concept for a new literacy of cooperation is symbiosis.

"...a mutually beneficial relationship that can evolve between different organism's in a system."

Symbiosis is not an instant thing, one doesn't issue a press release and announce a symbiotic relationship with customers, partners or vendors. Over time, reciprocal actions build relationships, yet reciprocity is a hard problem.

In a Social Source world, symbiosis is created with the winning strategy for a game of "tit for tat."

1. Be nice - don't defect at the first opportunity.
2. Retaliate - defect if others do.
3. Forgive - switch to cooperation when your opponent does.
4. Be clear - always react in the same way to your opponent's behavior.

In a Social Source world, you create open source software and share it with the world (be nice). If vendors and consultants choose to use the software without contributing back to the community, you withhold engineering support, priority bug fixes, and custom feature implementation (retaliate). When a vendor or consultant changes their minds and starts contributing back to the community, actively support their success (forgive). And all the while, communicate what a Social Source value system and ecosystem look like for others can behave as is expected (be clear).

Already, CiviCRM is becoming both endosymbiotic and exosymbiotic in the Social Source ecosystem. Endosymbiotic means one organism is literally inside of another. This is the relationship between CivicSpace and CiviCRM with CivicSpace's 0.8.2 release. Neither piece of software is entirely "whole" without the other.

CiviCRM is also exosymbiotic with content management systems like Drupal and Mambo. The pieces of software reciprocate (track a common set of users), but are seemingly distinct and can operate entirely separately. Ultimately, as CiviCRM becomes the basis for donor management, advocacy, and case management applications, a network of symbiotic relationships will evolve.

One interesting lesson from biology is that parasitism drives rapid evolution. In a Social Source ecosystem, not everyone needs to or even should cooperate and collaborate. Individuals and organizations that adopt software by "defecting" in the game of tit for tat by not contributing code and innovations back into to the community.

Social Source Sync

Synchrony: the process by which patterned behavior is created among many individuals without conscious control.

In a Social Source ecosystem there are a lot of actors: developers (building software), hosters (providing software as a service to customers), integrators (modifying software for customer needs), customers (using the software), etc.

Traditional proprietary software vendor models try to coordinate those actors under a single vendor. Developers have to use the SDK (software development kit) approved by the intellectual property owner. Hosters have to pay the intellectual property owner royalties. Integrators are only available from "partner programs" organized by the intellectual property owner. At least some portion of the customer spend to use the software goes to the intellectual property owner.

Even in the nonprofit sector this command and control model is prevalent.

The rational for this command and control model is that the resulting system has rules, standards and is predictable... if no one is in control, that would be too risky.

The concept of Synchrony challenges this analysis. If you have a system of actors that is communicating and engaging in some type of rhythmic give and take, those actors will, over time sync up with one another.

In a Social Source world, software developers, integrators, hosters and customers all communicate with one another, ask one another to meet their need, and contribute innovations back and forth. Over time, this rhythmic give and take yields coordinated cooperative action.

...the combination of strong and weak links can create unexpected and spontaneous outbreaks of coordinated behavior across decentralized networks.

The partnership between the Social Source Foundation and CivicSpace labs to build and deploy CiviCRM is a good example of strong links in the network. Each organization also has many weaker links in the nonprofit technology sphere, the political sphere and the open source community.

This emergent Social Source ecosystem has already begun rhythmic oscillation... PicNet and CivicActions are starting to use the technology for customers and are increasingly communicating with the partners with the strong links (CivicSpace Labs and Social Source Foundation). Over time, if enough actors join the system, communicate and exchange innovations, unexpected coordinated behavior should start breaking out across the network.

We believe these yet to be discovered opportunities for coordination will create major positive changes in the field of social purpose technology for non-profits and NGOs.