I decided to use ChatGPT as a research assistant this morning. I know, here we go, already disrupting the labor market.

Except it really didn’t work.

I’m writing a biography about a State Department official in the early 20th century who wrote or implemented some of our notorious immigration laws. Since he was raised on a farm in Ohio, I need to know more about the chores an Ohio farm boy would have done in the 1870s and 1880s.

First I asked ChatGPT to “find firsthand accounts of life on an Ohio farm in the late 19th century.” It gave me a kind of cheeky answer:

I don't have the ability to browse the internet or access specific firsthand accounts. However, I can suggest some resources and methods you can use to find firsthand accounts of life on an Ohio farm in the late 19th century:

It then proceeded to suggest I try consulting archives, books, newspapers, museums, and family histories.

Gee thanks, ChatGPT. That’s pretty much what I was doing before I met you.

Well, okay, maybe the problem is me. I tried scaling back a little, just asking for books about Ohio farms in the late 19th century. It gave me a list of five books, a few of which sounded kind of interesting, like “Ohio’s King Corn: The Politics of Agricultural Progress 1850-1900,” by Paul Salstrom. Paul Salstrom is an agricultural historian, so this sounded promising (except that I’ve never seen a history that defines a period from exactly the middle to exactly the end of a century).

I tried to track down all five of the books on ChatGPT’s list. Unfortunately, two of them apparently don’t exist. I’m struck by how plausible they sound while being completely fake. (Experiments run by myself and colleagues to draft legal provisions have proven similarly wrong.)

Guess I’ll go back to checking archives, books, newspapers, museums, and family histories. I had considerably more success locating sources doing it the old-fashioned way.

For now, I’m not too worried about ChatGPT as job competition for anyone.