What was the error of the Nicolaitans?

They were not named after a man called Nicolas.  The Nicolas of Acts 6 is mentioned in a positive context.  We don't need to go outside scripture to determine what this doctrine was, it's deduced from the etymology of the name.

Nico-, combinatory form of nīke, means "victory" in Greek, and laos means "people", or more specifically, "the laity"; hence, the word may be taken to mean "lay conquerors" or "conquerors of the lay people".

The fact that this error is mentioned in only two messages doesn't necessarily mean it's relevant to only those two.  Only Ephesus is specifically commended for rejecting it and only Pergamos is specifically criticized for having some who fully hold it.  It might be possible others Churches had a more in-between version of it.  It also seems like it may not have originated in either of these cities.

That's been my position in the past, but I'm prepared to change my mind if new information comes to my attention.

A lot of confusion about this issue comes from thinking it's the same thing as the doctrine of Balaam also mentioned in the message to Pergamos.  In the message to Pergamos Jesus talks about them following the error of Balaam (Pornea/Whoredom and eating food sacrificed to Idols), then says they also have some who hold the teaching of the Nicolatians.

This is why the opinions of the "Early Church Fathers" are not very credible on this issue, because right from the first of them to bring up the subject, Ireneaus, they are being treated as if they're the same.  This is a mistake I myself have made talking about the issue in the past, so it's understandable.

It's not till the Seventh Century people start saying the Nicolatians' error was that Nicolas let other men lay with his wife.  When Clement of Alexandria mentions this story he's referring to it as a positive.

However what if there is a third Church in Revelation directly relevant to this issue?

The second through sixth Churches in Revelation 2&3 are addressed as "The Church in _____".  But the first is addressed as the "Church of Ephesus" and the last as the "Church of the Laodiceans".

The city of Laodicea was named after a Seleucid queen Laodice.  The Greek roots of the name are Laos (people or laity) and Dike meaning either Justice or Vengeance depending on who you ask.  Notice how if you replace only one letter you get the same roots as Nicolatians.  That could support this name carrying a similar meaning, or maybe you could interpret it has having an opposite meaning.

Early Church Tradition says the first two Bishops of Laodicea were people named in Colossians 4:15-17, (one of them being a woman, Nympha, interesting), they could well have been before the major problems Revelation deals with emerged.  The third known Bishop of Laodicea was Diotrephes, a man refereed to very very negatively in the Epistle known as Third John, in fact he's been interpreted as being the first Monarchical Church Bishop.

And thus this all further backs up the premise of my Heresies of Asia Minor post, as well as The Gospel of The Beloved Disciple.

However the "victory over the people" meaning could also apply to Legalists, people who's issue was the polar opposite of the error of Balaam.  Which could justify equating them with Cerinthus or the Ebonites.