When I see someone with syphilis, it is very common for the person infected in an early stage (first or secondary) to have mild neurological symptoms. It may simply be a headache from being infected and feeling unwell, but many guidelines recommend ruling out neurosyphilis, which would require a lumbar puncture.
It is not a risk-free process, but if there really is a infection in the brain, the treatment requires very high doses of IV antibiotics because the penicillin we normally use barely penetrates the brain. At a recent conference, some researchers from the Germans Trias i Pujol Hospital in Catalonia explained that they are investigating a novel antibiotic from the oxazolidinone family that could be a useful tool in this situation.
Why? Well, these novel drugs can be administered orally, and they penetrate the spinal fluid well, and if they turn out to be effective in neurosyphilis, more than penicillin, we could use it in these cases without having to subject the patient to this very unpleasant procedure.
Ref: Ubals et al Repurposing drugs for syphilis. 34th IUSTI Euopean congress on Sexually Transmitted Infections and HIV, Bucharest, Romania.