Volunteer Data Entry works: 11,643 entries in Salesforce.com Central Repository at 7:13 AM up from about 7,000 at 3 AM

Add this to the 2,600 we did yesterday before we melted down the server and we have put about 14k forum postings into a database format with the help of volunteers. Good job folks!

Another Database? Aren't you part of the problem? NO!

The Red Cross has a missing/found persons database. We have identified another 10+ databases on the web. Web forums like craigslist are posting missing/found persons information there are probably 25 major ones of those.

We are consolodating all this information from all over the web. Hundreds of volunteers are entering craigslist and other forum entries by hand. 10,000 of them since yesterday.

We have published an open data spec, the PeopleFinder Interchange Format (PFIF), that can facilitate all the various databases syndicating information into a single database.

We are an open and community process working as quickly as possible to get information for individuals impacted by the hurricane. Please Join Us. http://socialsource.blogspot.com/

Hard core technology volunteers- hackers, Google engineers, super data heads are critical to the effort. Join the hard core techie email list by sending mail to katrinadev-subscribe@civicspace.org.

The hard core techie Wiki with the data standard is at http://192.122.183.218/wiki/index.php/PeopleFinderTech

3AM Central 9/5/2005. 10,000 records entered. Great Work!

3AM Central 9/5/2005. 10,000 records entered. Great Work!

The Katrina PeopleFinder Project is creating a central repository for all refugee records on the web.

Two main efforts are:
(1) VOLUNTEER
http://192.122.183.218/wiki/index.php/PeopleFinderVolunteer
Massively parallel volunteer data entry of refugee records into a central repository, conforming to the PeopleFinder Interchange Format (PFIF).
(2) TECH
http://192.122.183.218/wiki/index.php/PeopleFinderTech
Volunteer techies effort to publish the PeopleFinder Interchange Format (PFIF) and collect data from refugee databases across the web.

Great work!

http://socialsource.blogspot.com/

http://blog.social-source.com/ got slammed.

We are now at: http://socialsource.blogspot.com/

Thank you.

We need community leaders to step up.

We need community leaders to step up.

Self-organizing is hard. Thank you for investing your time. It will make a HUGE difference.

1. Pick an area.
2. Organize a leadership team on Omidyar Network (links below) [(3) is done already]
3. IM me so that I can publicize who the leaders for each team are and keep this blog post updated with the current information.
4. Leaders should post their contact information so that people can contact them.
5. If you can't lead, just subscribe to the email list (katrinadev-subscribe@civicspacelabs.org) and stand by while we get organized.

Keiran Lal (Kieran AT civicspacelabs org) will organize a conference call and IRC for latter to help with organizing.

To stay informed, send email to katrinadev-subscribe@civicspacelabs.org

If you are involved with the Katrina PeopleFinder Project, or want to be, it is critical that we improve our organization structure as soon as possible. Luckily, Jon Lebowsky has shown us the way. Thanks Jon! You can see how they organized the leadership team by following the Omidyar Network link for the volunteer data entry project.

Self-organizing is hard. Thank you for investing your time. It will make a HUGE difference.


The project has four main goals. Each on needs a few leaders and back-ups.

(1) DATA SPECIFICATION. Create, disseminate and support others in using the PeopleFinder Interchange Format.
Leader1: Ka-Ping Yee (ping AT zesty ca)
Leader2: (kleinpeterj at corp.earthlink.net)
Leader3: ?

(1a) Contact all existing structured data sites ask them to implement the data standard
Leader1: Katrina Data Project (?)
Leader2:

(1b) Help developers implement the data specification
Leader1:
Leader2:

(1c) Whatever else needs to be done.

Organize your leadership team here: http://www.omidyar.net/group/katrinarefugee/news/1/

(2) WRITE SOFTWARE. Write software that gets other people’s data into the PeopleFinder Interchange Format.

Leader1: ?
Leader2: ?
Leader3: ?

(2a) Assign folks to scrape specific databases
Leader1:
Leader2:

(2b) Create Salesforce.com central data repository.
Leader1: Jon Plax (jplax AT salesforce com)
Leader2:

(2c) Other items

Organize your leadership team here: http://www.omidyar.net/group/katrinarefugee/news/2/


(3) VOLUNTEER DATA ENTRY: Organizing a massively parallel volunteer data entry project to enter refugee data posted to online bulletin boards into a central database by hand.
Leader1: Jon Lebowsky (irc.freenode.net #globalvoices)
Leader2: ?
Leader3: ?

(3a) Don’t exactly know since Jon Lebowsky has it under control
Leader1:
Leader2:

(3b) Don’t exactly know since Jon Lebowsky has it under control
Leader1:
Leader2:

Your leadership team is already organized here: http://www.omidyar.net/group/katrinarefugee/news/0/


(4) Market the Katrina PeopleFinder Project and recruit volunteers.
Leader1: Andrew Hoppin (Andrew AT civicspacelabs org)
Leader2: ?
Leader3: ?

(4a) Recruit full, deep leadership teams for all segments of project
Leader1:
Leader2:

(4b) Other Items

Organize your leadership team here:
http://www.omidyar.net/group/katrinarefugee/news/3/

Gave money already? Give an hour of your time right now online. Katrina PeopleFinder Project

Refugees can search 20 web sites for lost relatives and still miss their entry on the 21st web site. There is a need to combine all the refugee data from big databases like Red Cross, large posting forums like Craigslist and many other sources on the web. The Katrina PeopleFinder Project seeks to create a single repository combining as many sources of refugee data as possible from all over the web without interrupting existing momentum.

We need help for both regular people and software engineers. Everybody is critical to building a central repository of ALL the refugee records we can find on the web. The Social Source Foundation, CivicSpace Labs and Salesforce.com Foundation are coordinating hundreds of people and organizations, including Craigslist and Earthlink.

Please consider giving us just an hour of you your time to do volunteer data entry. The PeopleFinder Project is seeking volunteers in four primary areas:

(1) Creating a technology specification for easily exchanging refugee information. A volunteer effort is working to assist online databases in implementing the specification.
Volunteer here (techies): http://www.omidyar.net/group/katrinarefugee/news/1/

(2) Coordinating volunteers that are writing software that takes information from online databases and putting it into a central database provided by Salesforce.com Foundation.
Volunteer here (software engineers): http://www.omidyar.net/group/katrinarefugee/news/2/

(3) Organizing a massively parallel volunteer data entry project to enter refugee data posted to online bullitin boards into a central database by hand.
Volunteer here (regular people): http://www.omidyar.net/group/katrinarefugee/news/0/

(4) Market the Katrina PeopleFinder Project and recruit volunteers.
Volunteer here (marketing folks): http://www.omidyar.net/group/katrinarefugee/news/3/

Katrina PeopleFinder Project

What is the Katrina PeopleFinder Project?
Refugees can search 20 web sites for lost relatives and still miss an entry on the 21st web site. There is a need to combine all the refugee data from big databases like Red Cross and large posting forums like Craigslist. The Katrina PeopleFinder Project seeks to create a single database combining as many sources as possible from all over the web without interrupting existing momentum.

How did the Katrina PeopleFinder Project start?
Community organizers from the League of Pissed off Voters, themselves refugees from New Orleans, needed a tool to help people in the New Orleans and Gulf Coast area stay connected to the communities they love. Enlisting the aid of nonprofit technology assistance providers Radical Designs, Social Source Foundation, and CivicSpace Labs, the site www.neworleansnetwork.org was created with open source technologies designed by and for nonprofits.

Realizing the PeopleFinder tool could be used to aggregate databases and information from around the web, the Salesforce.com Foundation joined the effort, combining corporate technology and resources with the power of the grassroots.

What is the Katrina PeopleFinder Project doing?
(1) Creating a technology specification for easily exchanging refugee information. A volunteer effort is working to assist online databases in implementing the specification.
Volunteer here: http://www.omidyar.net/group/katrinarefugee/news/1/
(2) Coordinating volunteers that are writing software that takes information from online databases and putting it into a central database provided by Salesforce.com Foundation.
Volunteer here: http://www.omidyar.net/group/katrinarefugee/news/2/
(3) Organizing a massively parallel volunteer data entry project to enter refugee data posted to online bullitin boards into a central database by hand.
Volunteer here: http://www.omidyar.net/group/katrinarefugee/news/0/

Who is involved?
The Social Source Foundation, CivicSpace Labs and Salesforce.com Foundation are coordinating hundreds of people and organizations, including Craigslist and Earthlink.

Contact
IM: dgeilhufe yahoo com

Links:
A blog to keep up-to-date (http://blog.social-source.com/)
Social Source Foundation (http://www.openngo.org/)
CivicSpaceLabs (http://www.civicspacelabs.org/)
Salesforce.com Foundation (http://www.salesforce.com/foundation/)
Craigslist (http://www.craigslist.org/)
Earthlink (http://katrina.earthlink.net/)

FINAL REFUGEE DATA EXCHANGE SPEC

Refugees can go to 20 different websites to find information on their loved ones. We are publishing a spec to facilitate data interchange among sites and that would allow the creation of a central database of most refugee databases on the web. We are also connecting database owners with volunteer programmers that can help implementing the spec. Special thanks to Ka-Ping Yee. Peoplefinder is a community effort lead by the Social Source Foundation, CivicSpace Labs and Salesforce.com Foundation.

The "official" data exchange format for refugee data is defined here:
http://zesty.ca/katrina/

If you run a refugee database, please publish your data via RSS in this standard. If you need volunteer programmers to help with implementation IM me at dgeilhufe AT yahoo com.

If you can volunteer to contact refugee databases (Red Cross, Gluf Coast, etc.) and help them implement the standard, please go here:
http://www.omidyar.net/group/katrinarefugee/news/1/

If can be a lead community organizer of programmers actually implementing the standard for websites, or for efforts to scrape databases, please go here:
http://www.omidyar.net/group/katrinarefugee/news/2/

Please diseminate this information far and wide.

Katrina: Gave money already? Give your time right now online.

A significant problem for refugees is that information on people is spread all over the web. If I was looking for a loved one, I could search 20 websites and still miss and entry about them. There are a number of technical solutions people are working on right now, but the old fashioned grassroots approach may be best.

Can you spare one hour today and enter missing persons data from forums accross the web into the peoplefinder database at http://www.neworleansnetwork.org/ ? We have people working on importing the major online databases on the web (Red Cross, Gulf Coast News, etc.) in addition to your data entry.

The first thing we need is community leaders that can help coordinate a massively parallel data entry effort so that thousands of volunteers can enter information from message boards across the web without too much duplication of effort.

The thing we need to resolve first is how to coordinate such a significant effort.

Go to http://www.omidyar.net/group/katrinarefugee/news/0/ where you will be coordinating the volunteer effort.

Please publicize this message far and wide.

Calling community organizers!!

We need some leaders to step in a organize some efforts. Are you up for it?
Email me dgeilhufe --AT socialsourcefoundation --DOT org

Goals we need leaders for:
(1) Organize a massively parallel volunteer data entry project.
(2) Organize programmers to populate a central database from the variety of databases on the web.

Katrina Refugee Database Data Standard

CiviCRM is being used for a refugee database at http://www.neworleansnetwork.org/

So we built a data standard that we are trying to get used in as many places as possible (to facilitate data exchange).

The current as final as we can get it dat standard is here:
http://zesty.ca/katrina/

Hurricane Katrina: Volunteer Data Entry

Information on missing persons is in hundreds of forums accross the web.

If a thousand people spend just one hour today entering information from forums into a database, hurricane victims will have a much easier time finding out the status of friends and loved ones.

We have a repository for this data at http://www.neworleansnetwork.org/ a grassroots website built with grassroots technology (CiviSpace & CiviCRM) by grassroots folks Radical Designs and League of Pissed Off Voters.

Step One:
Figure out how were going to coordinate a massively parallel volunteer data entry effort.
Sites that need to be entered by hand (we can't write software that will put their information in our database) are tagged in delicious.
http://del.icio.us/tag/peoplefinder+unstructured

Step Two:
Write clear instructions on how to participate.

Step Three:
Let as many people know about it as possible.

We are on step one. Send me an email at dgeilhufe at-- sourcialsourcefoundation dot-- org to help with the planning.

Current organizing locations/ how to help with central "distributed" database

OK, things are slowly taking shape.

First email coordination:
Our friends at civicspace labs have put up an email list:

If you are a techie/programmer, please drop me a line at david at-- socialsourcefoundation dot-- org and we'll get you on the mailing list.

Non-techies, please stand by :)

Second IRC:
irc.freenode.net
either #civicrm OR #civicspace
Coordination was going on there last night.

Screen scrapping targets

We want to get as many missing persons records into a single database as possible.

(1) Get techies to build software to "scrape" info from structured sites into a standard datamodel.

(2) Get volunteers to read forums and postings and manually enter that data into a standard datamodel.

Here are some of the places we need to have scraped/ entered into our standard datamodel. We'll of course be syndicating the unified database.

Lists of lists:
http://www.packrat-pro.com/katrina.htm
http://www.blessingsontheweb.com/

Red Cross has got their database up which is huge, so that is another good scrapping opportunity.
http://www.familylinks.icrc.org/katrina/people

The big list so far has been the Gulf Coast News site.
http://wx.gulfcoastnews.com/katrina/status.aspx

Our goal, again, is to combine all these databases into a single data model, offering a unified view of who is OK so refugees don't have to spend their time searching message boards and databases.

Ecosystems are not about design or control

Marnie Webb always triggers a good thought or two:

It’s about turning over part of yourself. In two essays (1, 2) Peter Merholz argues that it’s not about the technology. He writes: “Web 2.0 is primarily interesting from a philosophical standpoint. It’s about relinquishing control, it’s about openness, it’s about trust and authenticity. APIs, Tags, Ajax, mashups, and all that are symptoms, outputs, results of this philosophical bent.”

In a Social Source Ecosystem, openness, trust and authenticity are cultural norms. The actual code and behavior of actors are just "symptoms, outputs and results of this philisophical bent."

A Social Source Ecosystem: Catalysis

Catalysis-- an action or reaction among actors that is triggered by an outside agent--a very small amount of catalytic agent can facilitate a very large scale reaction.

Open source software becomes a catalytic agent that brings new actors into the social source ecosystem. Recently an online petition application was released using the CiviCRM API to store data. An entirely new community of users that care about online petitions will now be exposed to CiviCRM and potentially join our community and leverage our software.

One of the key factors for a successful catalyst is an environment conducive to broad cooperation across organizations, markets, commercial products, and human activities. In the software world, this means standards. TCP/IP, HTML and XML provide the basic technical standards, but for a Social Source Ecosystem, standard representations of actions, online donations, contact records, etc. Become just as important.

In the end, group forming networks are probably the most relevant mechanism for large catalytic impacts in the Social Source Ecosystem.







The implication is that the Social Source Foundation cannot be a central clearing house, but instead focuses on the basic rules and tools that allow any individual or group to immediately use CiviCRM for their purposes, creating a new group around their specific needs.

Social Source Socialist?

Oh, how I love it when corporations make arguements that can be applied in different ways. In the fundraising software space, there is a debate about people filing business process patents for specific online fundraising methods. The most noise comes from software vendors that will lose business if a single vendor is the only one able to offer a specific business process.

One arguement against business process patents is:

Shouldn't technology enable us to do more and to do it more effectively? Restricting our use of fundraising tools limits the number of people we can engage, the volume of donations we will receive, and ultimately, the universe of people we can help.

So I gotta do it:

Shouldn't technology enable nonprofits to do more and to do it more effectively? Restricting nonprofit use of fundraising tools (through expensive proprietary software licenses) limits the number of people nonprofits can engage, the volume of donations nonprofits will receive, and ultimately, the universe of people nonprofits can help.

This is not an arguement to make things free (the social source ecosystem depends on customer revenue), just an arguement to radically reduce barriers to adoption. ;)

A Social Source Ecosystem: Group Selection

In one of my early presentations on the potential of open source software for nonprofits, I had a slide called “Sounds like Socialism.” The slide tried to address a perception that a utopian view of sharing software and innovations simply couldn’t work. In our market-driven capitalist society, if you don't buy it, it can't be valuable.

The concept of group selection injects a decidedly cut-throat capitalism aspect into the concept of Social Source. The Institute for the Future notes,

“…groups work best when their members provide benefits to one another, but many of these prosocial behaviors do no survive through natural selection.” Individuals who effectively compete with other individuals succeed in evolution; those that cooperate are less successful.

How then do we conceive of a Social Source ecosystem? If it’s not one group of individuals and organizations sharing software in a utopian collaboration of nonprofits, what is it?

Group selection is the concept that individuals are not the only ones subject to natural selection—natural selection also operates at the group level—groups of individuals. Cooperation within a group can be a very important asset when competing against other groups.

This has implication for the Social Source ecosystem competing with the commercial software ecosystem, but also for the internal organization of the Social Source ecosystem.

Applying this to the Social Source ecosystem suggests there is no monolithic groups or single leader. Instead, some core principles (open source licenses) simply guide the entire system. Conflict and competition at a wider scale in the Social Source ecosystem will encourage local cooperation in order to compete in the wider group.

In the case of technology like CiviCRM, there is a strong incentive to support the creation of multiple donor database solutions, multiple volunteer management solutions, etc. The competition among those solutions will actually make those solutions stronger.

This leads both to competition among groups sharing volunteer management code for their specific solution, but also supports much broader cooperation as multiple volunteer management solutions share innovations at the CiviCRM level.

Things are getting curiouser and curiouser as I think through how a Social Source ecosystem might work.

Social Source Symbiosis

The Institute for the Future's second concept for a new literacy of cooperation is symbiosis.

"...a mutually beneficial relationship that can evolve between different organism's in a system."

Symbiosis is not an instant thing, one doesn't issue a press release and announce a symbiotic relationship with customers, partners or vendors. Over time, reciprocal actions build relationships, yet reciprocity is a hard problem.

In a Social Source world, symbiosis is created with the winning strategy for a game of "tit for tat."

1. Be nice - don't defect at the first opportunity.
2. Retaliate - defect if others do.
3. Forgive - switch to cooperation when your opponent does.
4. Be clear - always react in the same way to your opponent's behavior.

In a Social Source world, you create open source software and share it with the world (be nice). If vendors and consultants choose to use the software without contributing back to the community, you withhold engineering support, priority bug fixes, and custom feature implementation (retaliate). When a vendor or consultant changes their minds and starts contributing back to the community, actively support their success (forgive). And all the while, communicate what a Social Source value system and ecosystem look like for others can behave as is expected (be clear).

Already, CiviCRM is becoming both endosymbiotic and exosymbiotic in the Social Source ecosystem. Endosymbiotic means one organism is literally inside of another. This is the relationship between CivicSpace and CiviCRM with CivicSpace's 0.8.2 release. Neither piece of software is entirely "whole" without the other.

CiviCRM is also exosymbiotic with content management systems like Drupal and Mambo. The pieces of software reciprocate (track a common set of users), but are seemingly distinct and can operate entirely separately. Ultimately, as CiviCRM becomes the basis for donor management, advocacy, and case management applications, a network of symbiotic relationships will evolve.

One interesting lesson from biology is that parasitism drives rapid evolution. In a Social Source ecosystem, not everyone needs to or even should cooperate and collaborate. Individuals and organizations that adopt software by "defecting" in the game of tit for tat by not contributing code and innovations back into to the community.

Social Source Sync

Synchrony: the process by which patterned behavior is created among many individuals without conscious control.

In a Social Source ecosystem there are a lot of actors: developers (building software), hosters (providing software as a service to customers), integrators (modifying software for customer needs), customers (using the software), etc.

Traditional proprietary software vendor models try to coordinate those actors under a single vendor. Developers have to use the SDK (software development kit) approved by the intellectual property owner. Hosters have to pay the intellectual property owner royalties. Integrators are only available from "partner programs" organized by the intellectual property owner. At least some portion of the customer spend to use the software goes to the intellectual property owner.

Even in the nonprofit sector this command and control model is prevalent.

The rational for this command and control model is that the resulting system has rules, standards and is predictable... if no one is in control, that would be too risky.

The concept of Synchrony challenges this analysis. If you have a system of actors that is communicating and engaging in some type of rhythmic give and take, those actors will, over time sync up with one another.

In a Social Source world, software developers, integrators, hosters and customers all communicate with one another, ask one another to meet their need, and contribute innovations back and forth. Over time, this rhythmic give and take yields coordinated cooperative action.

...the combination of strong and weak links can create unexpected and spontaneous outbreaks of coordinated behavior across decentralized networks.

The partnership between the Social Source Foundation and CivicSpace labs to build and deploy CiviCRM is a good example of strong links in the network. Each organization also has many weaker links in the nonprofit technology sphere, the political sphere and the open source community.

This emergent Social Source ecosystem has already begun rhythmic oscillation... PicNet and CivicActions are starting to use the technology for customers and are increasingly communicating with the partners with the strong links (CivicSpace Labs and Social Source Foundation). Over time, if enough actors join the system, communicate and exchange innovations, unexpected coordinated behavior should start breaking out across the network.

We believe these yet to be discovered opportunities for coordination will create major positive changes in the field of social purpose technology for non-profits and NGOs.