Historically, golf's association with the Olympic movement has been decidedly bizarre. The First Olympics with golf were in 1900, and the last were in 1904. There have been talks about bringing it back for about 25 years but nothing has come of it so far.
The Games of 1900 were held in Paris. The previous year Captain Alfred Dreyfus of the French army, a Jew, had been convicted on a charge of treason. It was a bum rap and France was riven by the Dreyfus Affair.
With world attention focused on France for the games, the government wanted to demonstrate itself to be untainted by anti-Semitism. It dismissed Olympic game organizer Baron Pierre de Coubertin and his committee, some of whom were believed to be royalists and anti-Semitic, and handed responsibility to the Paris Exposition Company.
A contemporary historian remarked of the new organizers that "their intelligent appreciation of the task before them remains yet to be disclosed." The Olympics were doomed to chaos.
In 1904, Peggy Abbott, an art student and member of the Chicago Golf Club, was visiting France with her mother and both of them entered an international golf tournament at Compiegne.
Peggy won the nine-hole event with a score of 47. Her mother finished in seventh place. Such was the farcical organization that they understood this was the ladies' championship of Paris and went to their graves without realizing that Peggy had in fact become America's first woman Olympic champion. (info from Golf Digest)