Forum:Regarding importing GEDCOM data

Forums: Index > Watercooler > Regarding importing GEDCOM data

The Master Genealogist files

Several years ago (late 1990s), I did a bit of genealogical research using The Master Genealogist. Not sure how to quantify the amount of research.. perhaps 300+ names, but I honestly don't remember.

In the meantime, I switched to Mac (no TMG version on Mac) and moved to Japan (can't easily do any original research from here), and got "busy" with the rest of life.

BUT, somewhere, I have my old old TMG data, not in GEDCOM format.

Though I'm expecting a chorus of silence for an answer (heck, I'm not willing to spend the time on my own data! ;-) I wonder if anyone, as a labor of love, would like to

1. receive a copy of my old old TMG data
2. import it into your copy of The Master Genealogist
   • this would likely require update of the data to new format, but I think TMG will do this automagically
3. export it to GEDCOM
4. upload the GEDCOM data to this site
5. profit!

Oh wait, there's no profit involved, but lemme know if this sounds like fun!

If I recall correctly, Surnames include Nugen, Cole, Foley from Virginia area in 1800s and Trichel from Louisiana from similar timeframe.

Thunderrabbit 02:47, 10 August 2008 (UTC)

I use only Family Tree Maker. And I'm not sure of the latest developments in converting GEDCOMs here. (I noticed your edit summary saying you had trouble finding a page - the page with the trailing "s" in its name is the only "See also" link on the GEDCOM page.) Good luck! Robin Patterson 04:47, 10 August 2008 (UTC)

By 2013 maybe we are ready for more

The following started on my talk page. Links above, and the categories below, may help track down earlier discussions. I think User:Rtol was investigating the subject a while ago. -- Robin Patterson (Talk) 04:16, March 23, 2013 (UTC)

XML gateway to simple GEDCOM uploading (to start)

Robin, I see XML code at the bottom of person pages! This could be the key to importing GEDCOMs. I notice Special Pages:Import page is missing under Page tools on my Familypedia (only Export page is there). That's a disappointment. My Windmillwiki (it's out in the wild) has both Import and Export pages (I'm the bureaucrat). At my Windmolen (Dutch for windmill) site, where I am a bureaucrat, Import page appears... so it limited to bureaucrats.

I think I could write a program (or work with someone) to write one to convert a one person GEDCOM into XML. Then that XML could be imported into Familypedia to an unused person page. The Familypedia pages are only wikitext. Forms would be bypassed in this process, but the pages would be exactly like all the other standard person pages. GEDCOMs are just plain text that has tags in it. XML, as I perceive it, is souped up HTML, which is plain text. The GEDCOM tags would be switched to the appropriate XML tags. Special:Import page would do the work converting it from XML to wikitext.

At first the program would just produce a simple page with vitals and so on; later notes, sources, etc. be added to the translation process. Images are not included in GEDCOMs, so they would be left out. Being able to upload single person GEDCOMs to a new page would ease data entry considerably. Merging with an existing page could be worked out after the import process is worked out. For a while, the merging would be manual.

My reduced version tree has 7,015 persons, and 2,443 marriages in it, and it is growing. I don't think I will live long enough to get all that on Familypedia. My full tree has 67,069 individuals and 28,258 families... wasn't hoping to get all that online, though.

What programming language to use? I'm not sure. The conversion is really a very simple process given we are working with text files: the GEDCOM and XML files. All my programming for the school district was done in compiled Applesoft BASIC, and no one uses that anymore. Perhaps a modern scripting language like JavaScript (I've written some code in this language) could be used for the conversion from GEDCOM to XML. I know some word processors have macros that could be written to do the same thing in a word processor. A GEDCOM can be loaded and viewed in a web browser like Firefox. The JavaScript would be run from a browser. I think the PHP language could be used. I have written some PHP as well. Have the software to write and run it on my computer now. I've looked at Python and Perl languages , but can't say I know much about them. If code can't be run on Familypedia's server, perhaps it could be run somewhere else online to convert the GEDCOMs. For my own use, I was thinking about just writing a program so I could import the XML.

Multiple person GEDCOMs, and merging with pages that already exist is a bit beyond me at this time.

Has anyone ever tried this? Is there any code available from previous efforts. Is the person or persons that were involved in past efforts still active at Familypedia?

Although GEDCOM has a standard format, I know that is seldom strictly followed. This can be dealt with through experience, and a long period of testing before going public.

Familypedia is attractive because it may still be there long after I have passed on. WeRelate is permanent too, but is far more complicated to use than Familypedia. My database at Ancestry.com is permanent, but not readily viewed by the public, and even difficult to find for Ancestry subscribers.

You can do whatever you want with this text...I don't know exactly where to put it myself. Whroll (talk) 04:06, March 23, 2013 (UTC)

GEDCOM is showing its age, I believe, and there are lots of variants and maybe not even a next edition. I should check the latest version of Wikipedia pages before we get too deep. -- Robin Patterson (Talk) 07:57, March 23, 2013 (UTC)

Here's some more on the subject:

• Forum:Gedcom bot and Rtol's improvements to Yewenyi's process. Whroll (talk) 05:28, March 23, 2013 (UTC)
• Help:Loading Gedcoms Whroll (talk) 05:42, March 23, 2013 (UTC)

PHP or something simpler?

(another section and first response) from my talk page that needs to be here; slightly edited)

Question:Showfacts person template

I hope you're not too far behind. Our internet here is down now and then, but it is improving with time.

I had a question as I prepare to use PHP to insert information into the table produced by the Showfacts person template. I don't fully understand the more complicated templates, but it is not necessary to know the inner workings of templates to get GEDCOMs converted. I notice that only data types (e.g. "birth_year=", "death_year=") that contain data are displayed. Persons have varying assortments of data types. Can additional data types be added, or is the table limited to a particular set of data types? For example, could I add "residence_year=" and "residence_place=" in the table? If the set is limited, where can I see the complete list? The PHP program needs that information, all the data type names, included in the code.

I have written successful computer programs using complex text string manipulation before. I think PHP can handle the job of GEDCOM to Wikitext conversion. It is more highly developed now than it was when I last fiddled with it.

I'm not sure yet if it can be a one or two or more steps operation. As I see it, the user would upload the GEDCOM, with a maximum allowable size. GEDCOMs could be upload as a file using the existing Upload photo link on the left menu. It's not just photos that can be uploaded. Permitted file types listed there on the upload page include ".ged" (GEDCOM). The GEDCOM would be viewed by the arbitrator and approved or disapproved. There are tools to browse GEDCOMS and edit them without loading them into a regular genealogy program. Even Microsoft's Notepad program can accomplish that. Once approved, the GEDCOM would be uploaded to a more open web server, where PHP would be used to convert the GEDCOM into individual Wikitext tables, along with whatever else is needed for each standard person page. Then the pages would be uploaded by a bot (an automated "user") into Familypedia. I'm almost certain Wikia allows bots and limits them to Familypedia's normally accessible content area.

I don't think Wikia will ever give us permission to run PHP at Familypedia. It is too powerful, and therefore a security risk for them the way they have the various wikis on their system set up. Normally a Mediawiki sysop would have access to their wiki's root directory. Familypedia sysops most likely would not be trusted by Wikia to access root. An error would have disastrous consequences. But that will not stop development of GEDCOM conversion, we still have access to the part of the wiki that we need to get the job done. I hope I am not too long-winded. Whroll (talk) 01:46, March 26, 2013 (UTC)

That is absurdly over-complicated: I recommend (again) my procedure: export the GEDCOM from your database (I use PAF) and process it on your own machine (I import it into LifeLines and run a script to output the page, then tweak it with some vbscript). Then you can just create the page on the wiki (or import, if you want to do them in bulk). I find that the fiddly bit is getting the place names correct. Thurstan (talk) 03:30, March 26, 2013 (UTC)

The placenames can mostly be handled by increasing our rate of redirect-creation, I guess. -- Robin Patterson (Talk) 03:43, March 26, 2013 (UTC)