Policy at scale: What makes an actionable complaint?
Here’s something that you may not know about abuse desks: they’re busy. The further up the “food chain” that you get, the smaller the relative number of complaints that you’ll see, but almost any abuse desk is going to be dealing in at least several thousands of complaints per year.
Dealing with complaints, then, is somewhat art and somewhat science. There are questions that need to be resolved and issues to be handled. This post will talk about the questions to be resolved when considering whether a complaint is even actionable.
Questions to be resolved
When a complaint is received, the first and most obvious question that has to be resolved is “Is this actionable?” We have to recognize the fact that not every complaint specifies a violation of policy. Additionally, not every complaint contains enough detail to make it actionable.
No violation of policy
Here’s an illustration from my very first job in policy enforcement: One day we received a spam complaint about a customer that they were sending mail that the person had not opted in to receive. Here was the content of the message that they were complaining about:
Thank you for submitting your resume to us regarding the (name of position) opening. Unfortunately, we have decided to move in a different direction with filling this position. We will keep your resume on file and will be happy to let you know if a different position for which you may be better suited arises in the future.Paraphrase of an email that was complained about
I’ve often said that had this email had different content (“Congratulations! We would like to offer you a position with our firm paying $250,000 per year plus benefits and options.”) that we never would have seen that complaint. And, “content that the recipient doesn’t appreciate” is rarely going to be considered a violation of any decent provider’s policies.
Not enough data to be actionable
Generally speaking, email service providers want to see the body and unredacted headers of an email. That makes for a well-formed and fully actionable complaint. With that information, we can:
- Establish that the message came from a system over which we have oversight
- Determine which customer sent the message (in other words: who, exactly, are we investigating?)
- Determine that the message was sent within an actionable time frame.
Sometimes, server log files are extremely appropriate as evidence, too. This is especially true in the case where a customer may be sending messages to many non-existent addresses. Since those messages will all bounce, there is literally no headers to tender, but the data in the log file can give me the same bits of information and provide a good springboard into an investigation of a customer’s practices.
In this context, it really does help to think of policy enforcement like you do the police. You can’t go to the police and tell them about the bank robbery that you think was committed in a completely different country half the world away. That’s simply not within their jurisdiction. And while they might be able to assist with getting you in contact with the right person to handle a complaint about an issue the next town over, the further away from their jurisdiction that you get, the less likely that you should assume that they will be able to assist you.
The least actionable complaint
The least actionable complaint is the one where the person writes in and says “You have a spammer on your network,” “Customer_name is a spammer,” or “Customer_name is violating some_law” without any proof. Policy enforcement does actually want to know all of those things, but our currency is evidence and this is a request for me to task resources to the generation of evidence based merely upon the assertion that a particular customer has (or is) a problem. Unfortunately, I don’t know if the problem is with one list or if the customer’s entire business model has them operating in violation of my policy. If it’s only a single list, how do I find out which one? When I can start from a single, known list, I can work my way backward from smaller questions to larger ones.
So, when the issue comes down to “devote resources to go on a fishing expedition” or “devote resources to investigate complaints with data” it’s a really easy call to make.
The “GWF” and threshold issues
I almost labeled this one as the easiest to close. But, until you get used to them, they take a level of investigation and can cause confusion. The acronym “GWF” stands for “Goober With Firewall.” It refers to complaints that come in from people who generally don’t know how servers work with log files attached to the complaint showing normal network transactions occurring. Usually, these will come in with breathless notes saying that a mail server has been attacking a name server on port 53 by looking up MX and A records. (Hint: That’s how a sending mail server discovers where to find the receiving mail server.) Or, the breathless note will talk about a mail server attacking another mail server by connecting to it on port 25 and sending mail. (Hint: that’s what mail servers actually exist to do.)
If the issue is that a server is occupying all available resources, then that should be stated. If the issue is that a server is sending mail too quickly, or is opening too many simultaneous connections, or some other issue, then that should be stated. But, if all that you do is tell me that mail servers are doing things that mail servers are supposed to do, then I’m going to close that case immediately because it does not show me anything that’s actionable.
A word about time frames
This one is more of a matter of taste than anything else. Abuse desks deal in large numbers of cases and that’s important to remember. Customers who were doing bad things a year ago have either learned their lesson and changed or they’re still doing bad things today and are continuing to get new complaints. That places a freshness bound on incoming complaints.
Depending on the needs of the abuse desk and how busy they are overall, this number can be quite elastic. My own thought on this is that all other things being equal, I want to start an investigation into a mailing within two weeks of the time it was received by the person complaining. In my experience, when you push out past about a week, customers start asking “why did it take so long to contact us about this?” You can generally get another week by pointing out case volumes and backlog. But, the complaints become more vociferous when you get past that two-week point and you’re spending more time defending the lag than you are investigating the case. And, as the saying goes, “spammers gonna spam.” So, there will always be more spam, if there is a genuine issue.
Nevertheless, I feel compelled to note that I know of other desks who operate on much different scales. One has the headcount to set their marker at around 4 days. Another actually pushes things out to a month.
When working at scale, an abuse desk depends upon complainants to provide enough information to prove — not merely assert — that the actions complained about
- Recently happened
- From a machine that the abuse desk has jurisdiction over
- As the result of an action that an identifiable customer took.
That information usually takes the form of a message with routing headers. But, it’s also possible to provide a log file if a sufficient explanation is included to say how or why the network traffic depicted is not normal traffic.