Security at Startup

Table of Contents

In my opinion, security is one of the most forgotten aspects of software engineering. It rarely gets focused on until it’s too late. Even though at least one incident lands on HackerNews every week where some data gets leaked or someone gets hacked — people still think, “Nobody cares about my little startup.” You might think you're too small to be noticed by the big, evil hackers. Wrong. Size doesn't matter. You're always a target; there’s always data to leak and ways to exploit your business.

That should always be your assumption as a young CTO or an early-stage engineer. Imagine you have Monitoring set up, and you get an SMS late at night that your service is going up and down constantly. Something feels weird. You get up, check the logs, and there are different IPs hammering your login endpoint. You think, “Brute forcing login? That’s so 2000s”, but then you notice that there are successful responses to those login attempts. Congratulations. You fell victim to credential stuffing. What awaits you after this realization is a battle with the bots and disclosure to your users that their data was leaked.

How credential stuffing works, for example, is that someone purchases leaked databases with email/passwords and then expands the data stored in that database with new data from other different services where the victim might’ve been registered. In the end, the attacker ends up with a lot of data about a user, which can then be used in other malicious ways.

Disclaimer: I’d like to mention that I’m not a security expert. I did some security courses and penetration test certificates back in the day, but that didn’t make me an expert. I’m more of a generalist who had to deal with many different scenarios in my career, which I’d like to discuss in this essay. Please tell me if you think I’m wrong or if something can be improved. P.S. None of the mentioned companies/tools paid to be in this article.

It’s no surprise that Google and others invest heavy resources into bounties and white-hat hacking — the cost of a breach is much much higher for them.

🏄 But then again, not all of us are Google with deep pockets for security. But you don’t have to be to run a tight ship. You can do many things to secure your startup that doesn’t require millions of dollars.

So we will be looking at cheap man’s security when you’re starting out from your garage or just getting some traction. And keep in mind there are always at least two components you need to be keeping secure:

1. Code & Infrastructure. This is all about ensuring that the software you built that is exposed on the internet does not fall victim to cyber-attacks. A typical example here is people poking holes in your firewall to try and steal data or use your infrastructure as part of a botnet.
2. Yourself/Employees. This is all about ensuring that nobody gets access to your business without you knowing and allowing that. One typical example usually targeted at a company and not at the software is phishing emails and social engineering.

Keep in mind - no system is secure. If humans built it, it can be broken into by humans.

Phase 0: You just did a beta launch

First of all, congratulations on shipping your app; it’s a huge milestone. Now, let’s talk about everything that can go wrong here and how to protect yourself.

Error Tracking is first on our checklist. It’s not directly security-related, but it is definitely VERY useful to have something like Sentry or Datahog properly set up from day 0. This helps quickly identify and address potential vulnerabilities or bugs that could be exploited.

Database Backups. Your database should be a separate instance and set up in a redundant way. You will thank me later once something goes horribly wrong and you frantically search for the latest stable DB snapshot. Point in Time Recovery, which Google Cloud offers, is a life-safer that literally saved me from having gray hair in my 20s.

Never trust the payload. This includes validating and sanitizing all inputs to prevent ANY injection attacks. If you’re building a multi-tenant system - make sure that all your requests and database queries include a userID or organizationID check. You can test your endpoints by switching up the IDs. Over the years, I’ve seen many listing endpoints that didn’t filter the data correctly. One of the most prominent examples that stands out in my memory was an audit I was part of, where the organization’s data was secured only by obscure UUIDs. The assumption was, “You don’t know the other organisation’s ID, so obviously, you can’t fetch their records.” Well, they were proven wrong.

As an example to the above case — if you have some Organization/Company/Business Entity structure that stores documents and you give the users of these Organization Entity the ability to fetch them by ID, make sure users from other organizations cannot fetch them by simply knowing the organizationID and documentID. Always filter the results based on session data first and only then on the payload data.

POST /user/<organizationID/documents/<documentID>

In this example, regardless of what’s provided in the payload, we first check if the logged-in user has access to the organization. Then, we check if the logged-in user has permission for the document. And only then can we return the document.

Speaking of security through obscurity, in my opinion, it’s underrated. Yes, it doesn’t give you THAT much protection, but it definitely adds a layer of complexity that will weed out some bad actors. For this — having at least two types of IDs in the database it makes it a tiny bit harder to reverse engineer:

1. Internal auto-increment integer type - you can use these in your backend code for internal references and foreign keys.
2. External UUID type - you can use these for external communication in your web app to make enumeration impossible and sometimes confuse bad actors.

If you’ve just launched your app, tons of bots will try to figure out what you are, and if you’re a loser, that can be hacked easily. In this case, The easiest thing you can do is use Cloudflare, Google Armor, or AWS Web Application Firewall, depending on where you’re hosted. Cloudflare is most effortless to set up as you switch the NameServers, pay 20$ per month, and flip a switch. This adds another layer between your server and the users; in some cases, it makes it harder to fetch the real IP, making it harder for attackers to figure out where to look for exposed port 22. Not only that, but this will allow you to grow without much of a headache as it protects you from the OWASP Top 10, DDoS, and weird bots. I very much recommend doing this. Once you’ve grown some more, you can invest resources into setting up complex rules, e.g., Rate-Limiting login attempts based on risk factors.

Also, if you’ve just exposed your app to the internet, you will probably start receiving the so-called “beg bounties”. The email subject will say something like “WARNING: SERIOUS CLICKJACKING VULNERABILITY FOUND” or “VULNERABILITY: NO DMARC RECORD ON YOUR WEBSITE.” They’re usually harmless and relate to public-facing configurations that aren’t that critical. You can read more about them here and here. I see them as a type of “racketeering” similar to the “I found your password which is {insert outdated password here}, pay me bitcoin”. Low-effort spam emails get annoying quite quickly, so be sure to add “X-XSS-Protection” and “X-Frame-Options” in your Nginx/apache setup to avoid the hassle.

I probably don’t have to remind you, but I still will; none of the services used by your application should be exposed outside of your private network, and even inside the private network, you should tightly control which service talks with what service. Since you’ve just launched, the services landscape is probably not that big, but it is still helpful to keep this in mind as you add more and more services.

🏄 Suppose you have an Admin Area where you manage your users — set up IP whitelisting. It’s a no-brainer and gives you peace of mind.

Don’t write your own: cryptography, authorization, rate limiting, file processing, payment gateways. The slightest error or oversight in these cases can lead to significant vulnerabilities. Cryptography is littered with examples of 'unbreakable' systems being cracked due to minor flaws. Writing your file processing routines can expose your system to various risks (especially when dealing with PDFs), including buffer overflows and injection attacks. So, think twice before you allow users to upload files.

Add as much logging as you can. Writing logs is cheap; you should start doing that as early as possible. If your application gets hacked and exploited somehow, you better have a trail in your logs that will stand out to you so you notice earlier rather than later. The logs you write now will eventually evolve into log monitoring with alerts, but just writing as much as you can is enough at this point.

At this point, I will assume your startup is a two-person show — one technical co-founder and one non-technical. So, how should you protect your company at this point? Well, you need to think of yourself as a user. I mean, somewhere you ARE a user, somewhere in the darknet, there’s a profile on you with the services you’ve used, with all your known emails, brute-forced hashes, and even clear-text passwords.

Let your founders know that from now on, you’re all using Password Managers - no more passwords on the Post-it notes or sharing any accounts. After that, you need to secure your internal communication. How do you know that the message sent to you from your co-founder, “Hey Justin, really urgent. Can you send 10’000 USD to this IBAN? I will call you back later.” Is it genuine and not some scam/phishing attempt? You might think it’s funny that I bring up such an example, but this is a working and very active way of phishing. Even META (ex-Facebook) fell victim to a similar scam.

So, now that we’re upping the game — agree with your founders about which “workspace” provider you will be using and which communication tool will be the “official” channel. I’m primarily familiar with Google Suite, but there are others. Our company has Slack for day-to-day discussions and email as the “official” communication tool. Here’s the Todo when setting up the workspace:

• Start setting up 2FA with the Authenticator App ( Don’t use the SMS one. It’s very risky) and set it as mandatory for everyone.
• Set up Gmail with your domain and verify it via DNS.
• Add SPF, DKIM, and DMARC Records (To avoid beg bounties, and anyways, you only need to do it once, so it is worth the hassle)

If you’re still making business decisions on WhatsApp — add a PIN. Or better yet, completely move to some Corporate Messenger like Slack or Teams. Also, set up 2FA.

In these early days, the rules you set up now aren't just temporary placeholders. They're going to stick with you, grow with you, and if they're half-assed, they're going to bite you in the ass later. The stuff I listed above should be enough at this point. Let me know in the comments if you think I missed something; I will add it, of course.

Phase 1: You hired your first employee

So we have all the basics, and you’ve grown quite a bit. Congratulations on that. At this point, you need to focus more on securing everything around the app — help desk, laptops, VPNs, Deployment processes, etc.

For example, if the first employee you hire is a support agent, they need some way to take action based on the user’s problems, meaning they need tools to access personal data or manage user accounts. From this point, they become a security vulnerability #1. To give you perspective on how bad it can get — there was a security incident at Twitter in 2020 where a person managed to get access to their support dashboard via a complex phishing + social engineering. This dashboard allowed them to gain control of any account.

Enjoyed the read? Join a growing community of more than 2,500 (🤯) future CTOs.

Email address

Subscribe

Free, no spam, unsubscribe at any time.

As you’re growing in terms of employees, partners, and vendors, you must ensure the contracts with them are proper — solid data processing agreements, non-disclosure clauses, Service Level Agreements, and Privacy Policies. I can guarantee that some of your clients will be very privacy-sensitive and ask many questions about your vendors and practices, so you better prepare for that early. So, sit down with each vendor and ensure you’re aligned on how the data flows between your services. It's not just bureaucracy; it's protecting your back, especially if you’re in Europe.

Let’s talk about steps to get you “more” secure. As you can see, I’m not saying you will be 100% protected, as that’s impossible, but this will put some hurdles in the hackers’ way.

I assume you’re working remotely, which is the most popular way after the pandemic. Implement Virtual Private Networks (VPNs) for secure remote access to your infrastructure. This process was a hassle to set up back in the day, but nowadays, there are fantastic tools that make it straightforward to integrate into your business flows. I recommend using TailScale. It also helps you restrict administrative and management access to sensitive systems via ACLs.

Employee Laptop and Device Security: Encrypt the hard drives. Implement automatic sleep mode for laptops and require password re-entry upon waking. This helps protect data if a device is left unattended. Assume your employee will lose the computer. I’ve had cases where employees lost their MacBooks in the first two weeks of working.

It’s probably a bit early for the MDM Policies, but keep an eye on it for the future.

Now, let’s talk about securing your development practices. I’ve recently learned these practices even have a name — DevSecOps. Fancy word, which basically means that security is a continuous part of development, deployment, and maintenance.

🏄 Your master/main branch should be protected from direct pushes.

We’re going into the automation part here. The goal is that stuff gets checked for human security factors without you consciously making an effort. If you’re utilizing Bitbucket/Github/Gitlab — you can set up actions that process the code before it gets merged into the development branch. Here are some examples that can help you validate your code:

1. Static Code Analysis — one of the popular ones is SonarCube
2. Linting - your choice.
3. Some tools check for accidentally leaked API Keys, committed secrets, and credentials.
4. Package vulnerability analysis - Snyk analyzes your packages to see if there are any known exploits. For my taste, it’s too panicky, but it does its job well overall.
5. But wait, there’s more, many more

At this point, you might also want to make sure that each deployment can be referenced in your monitoring tools, e.g., when a new release is deployed, the SHA hash of the commit is sent over to Sentry/Datahog and attached to the environment variables. Hence, we know which version of the app the instance is running. This helps understand where the bugs/logs originate from.

To have peace of mind, I’d recommend subscribing to security mailing lists to immediately get notified if there’s another 0-day exploit that’s running wild. Sometimes, you will see some security issues pop up on HackerNews, but those are usually the big ones, like vulnerabilities in OpenSSL or Intel CPUs.

Phase 2 and beyond

So you might ask, what’s next after you’ve done all the basics? Well, it’s pretty simple: as your startup grows, the role of a Chief Information Security Officer (CISO) becomes increasingly important. You must delegate this to someone who knows what he’s doing and can take responsibility for that. I’m not saying that you can’t do this job, but you shouldn’t be the one doing it if that’s not your core competence. This isn't about playing the Tower Defence game with black-hat hackers; it's about a single person, someone you trust with security, who builds a defensive culture at your startup that is also aligned with your business objectives. He will become the person you come to when you need answers.

After you have a person who does this full-time, they will need to hire security experts, networking experts, and many other roles. The complexity of managing such roles and permissions increases. At this point, your company will change — you will start having badges to get in and out of the office and have preinstalled notebooks distributed to your employees with remote controls and kill switches.

The more you grow, the more certificates you’ll need to show — SOC2 and ISO27001, to name the most popular ones. These aren't just badges to enhance your company's profile; they are comprehensive, and I mean very comprehensive, guides to building a secure and trustworthy business. Everything is rigorously documented; you just need to follow the red line and then show an auditor that you followed that red line. Nonetheless, this is much more efficient than filling out a questionnaire for every big client you work with that handles sensitive data.

Regular audits and penetration testing will become a day-to-day business for you. At some point, you will have red teams inside the company itself who will be trying to break the software constantly. Google has a whole department to find vulnerabilities in Open-Source libraries, and they do find a lot.

You’ll start a bug bounty program where you payout X to ethical hackers who are making a living by finding holes in everyday software. It allows you to harness the collective expertise of millions of ethical hackers to uncover and address vulnerabilities.

If you get to this point, my advice is probably useless, you'll have someone much more experience than I who can answer your questions, but I’d like to think I provided you with some overview of the road to this point. Your security measures should also scale as you scale, evolving from basic defenses to sophisticated, multi-layered systems. Security is not just a Startup issue; it's a fundamental aspect of any business. Better Safe than Sorry.

I wish you a secure and successful startup journey.

Other Newsletter Issues:

• Habits of great software engineers
• Networking as an introvert CTO
• Database Migrations
• Proper Documentation