Another tall tale (there's no reason to believe it's anything but) dates the origin of ladies' privilege to the 5th century, around the time (speaking of tall tales) St. Patrick drove the snakes out of Ireland. As the story goes, St. Patrick was approached by St. Brigid, who had come to protest on behalf of all women the unfairness of having to wait for men to propose marriage. After due consideration, St. Patrick offered St. Brigid and her gender the special privilege of being able to pop the question themselves one year out of every seven. Some haggling ensued, and the frequency ultimately settled upon was one year out of four leap years, specifically an outcome which apparently satisfied both parties. Then, unexpectedly, it being a leap year and St. Brigid being single, she got down on one knee and proposed to St. Patrick on the spot! He refused, bestowing upon her a kiss and a beautiful silk gown in consolation.
We may conclude, among other things, that St. Patrick was better at dealing with snakes than with women.
Earliest English-language sources
The American Farmer, published in 1827, quotes this passage from a 1606 volume entitled Courtship, Love and Matrimonie:
Albeit, it is nowe become a parte of the Common Lawe, in regard to the social relations of life, that as often as every bissectile year doth return, the Ladyes have the sole privilege, during the time it continueth, of making love unto the men, which they may doe either by wordes or lookes, as unto them it seemeth proper; and moreover, no man will be entitled to the benefit of Clergy who dothe refuse to accept the offers of a ladye, or who dothe in any wise treate her proposal withe slight or contumely.
That the reversal of gender roles was well recognized as a leap year motif by the beginning of the 17th century is reaffirmed in this passage from the Treatise Against Judicial Astrologie by John Chamber, dated 1601:
If the nature of anything change in the leap-year, it seemeth to be true in men and women, according to the answer of a mad fellow to his misstress, who, being called knave by her, replied that it was not possible, "for," said he, "if you remember yourself, good mistress, this is leap-year, and then, as you know well, knaves wear smocks."
It's alluded to again in this couplet from an Elizabethan-era stage play called The Maid's Metamorphosis, first performed in 1600 (a leap year):
Master be contented, this is leape yeare,
Women weare breetches, petticoats are deare.
Finally, we would be able to push back the earliest documented reference to "ladies' privilege" an additional 200 years if only we could authenticate this couplet attributed to Geoffrey Chaucer (c. 1343-1400) by Vincent Lean in his Collectanea, published in 1905:
In Leap Year they have power to chuse
The men no charter to refuse
Unfortunately, the only other source I've found it in is The English Year by Steve Roud, who notes that the attribution has so far proved "impossible to verify."