ZidShip: From 0 to 100k Shipments (the Product and the Engineering)

النسخة العربية : هنا

In April of 2021, ZidShip has reached 100K delivered shipments, so we think it is time to share our experience of launching such a new and unique product.

ZidShip is a shipping aggregator. We work with tens of courier partners to provide a standardized set of service levels with unified pricing. So, to the merchant who uses ZidShip, they are dealing with ZidShip, they are only signing ZidShip's contract, but they are getting the coverage of the 15+ couriers all at once. What makes ZidShip's model unique is that we are at the forefront of operations. Our system intelligently chooses the proper courier, and our team handles the merchant's accounting directly without referring to the individual couriers.

In this post, we will be presenting the story of ZidShip. How it went from being a good-to-have service that we should work on in a couple of years to a necessity that should be launched in a couple of months. We will be discussing both product and technical challenges we had to deal with and the decisions we had to make.

What was wrong?

Until January 2020, the market was fragmented between the small couriers struggling to pull their operations together and the large, well-established couriers not adapting to eCommerce. Small couriers could not provide a consistent service (nor sufficient coverage), and large couriers were either not competitively priced or unwilling to deal with smaller merchants. And what caused even more trouble was the COVID-19 pandemic. As a result, all couriers, large and small, could not handle the spike in demand and the constantly changing regulations and health requirements, which in turn has caused all sorts of frustration to the merchants and their end customers.

How do we launch fast?

Given the pandemic circumstances, we knew that we need to launch as soon as we can to get feedback quickly and iterate on our original model.

The first obvious technology question that was asked is the common "should we buy a system off the shelf or build our own?"

We knew that the most time-consuming area to build is courier integrations. We were planning to launch with at least ten couriers, so we will have to integrate with each of them (you can imagine the chaos in the docs and inconsistency of these external systems). Thankfully, courier integrations is the area where most of the solutions we looked at were fulfilling.

On the other hand, after examining the available solutions, we found that most of them do not conform with our operational requirements. For example, we have a relatively unique accounting procedure, and that our team is at the forefront of the operations, unlike most of those solutions providers. We also knew that these requirements would change as we tune things, and relying on an external vendor to deliver those enhancements may not be the most time-efficient approach.

So, should we build our own system? Yes, and no. We settled on buying a system for its courier integrations and wrapping it with our own system that does everything else. From rate calculation, routing, tracking, and bookkeeping.

This approach allowed us to launch quickly and only build exactly what was needed at the moment.

As part of our goal to be lightweight and autonomous, we built our system independently from the core Zid system. We used Python and Django with Postgresql – as opposed to PHP/Laravel and MySQL. That allowed us to utilize the wide variety of mature open source libraries and tooling available in the Python ecosystem to accelerate our progress.

Experimenting

Uncertainty is tightly coupled with any product launch. You never know if your assumptions about the market and customer's behavior are correct. We deeply believe in that principle. Our approach is never to assume too much. We started with the most basic assumptions to test the market and built on top of it. The center of our offering is the concept of Service Levels, where we divided destinations based on the speed of delivery (consequently, dividing the couriers too). We started with four levels, creatively named Level 1, Level 2, Level 3, and Level 4. With on-demand same-day delivery, next-day delivery, 3 - 5 day delivery, and 5 - 7 day delivery as the delivery speeds, respectively. And to ensure that we can provide a consistent service, we decided to only launch for merchants in one city (Riyadh). Moreover, during the first few months, we did not even have Level 1 available (we will discuss this in more detail in the next section).
Launching in one city allowed us to focus on validating our assumptions and providing a consistent service to our customers.

Few weeks after launch, technical problems started to appear. The vendor's system that we purchased started to fail due to changes implemented by the vendor. Consequently, our system failed too, given that we rely on them to decide if the customer's city served or not. A few days later, we had a similar incident with printing waybills. We realized that we could not continue to rely on any external party in our critical paths, namely, receiving orders and generating waybills. Therefore, we decided to reimplement the outsourced components gradually. From the core Zid systems perspective, nothing has changed during this process because we were wrapping this system with our own, so users of ZidShip were unaware of this change (they noticed the increased stability).

Launching ZidShip Now (Level 1)

Since ZidShip's conception, we knew that we must launch an on-demand delivery service. However, we did not launch as part of the initial offering of ZidShip because it is fundamentally different than the rest of the service levels. In on-demand operations, everything has to be real-time. We cannot wait until a pickup is scheduled or enough orders are picked up to dispatch delivery. Furthermore, we did not yet understand the operations enough to automate it.

Going off our principle of carefully choosing what to build and how to build it, we launched ZidShip Now with a semi-manual operational model. Once orders arrived at our system, a joint operations team from the pilot courier and ZidShip staff took care of dispatching and following up with the orders. This manual flow gave us great insights into the operations and the issues that needed automation. Based on this knowledge, we built a fully automated flow and added more couriers. And more importantly, set the level of quality and expectations we have for those couriers on this specific service level.

Launching Dropoff service levels

Three months after launch, we discovered a new type of merchants. Merchants with low/medium demand and high-cost sensitivity. These merchants were fine with dropping off the shipments at the couriers' office instead of having the couriers come pick them up. On the courier side, the courier would drastically reduce their prices if they did not have to dispatch a pickup trip.

Based on this merchant persona, we launched the Dropoff service levels with the three major couriers in Saudi: Aramex, SMSA, and Saudi Post (now, SPL). Dramatically decreasing the cost and significantly increasing the coverage – we are no longer constrained by pickup scheduling complexity. Now, wherever there is a branch for any of these couriers, ZidShip can ship from there.

Scaling

Scaling can be divided into two related categories, operational scaling and technical scaling. Operational scaling is expanding coverage of cities and regions while maintaining service consistency. It also includes managing more couriers and ensuring that they are complying with the Standard Operation Procedure (S.O.P). On the other hand, technical scaling includes supporting the operational scaling in things like integrating new couriers and helping identify non-complying couriers (through analysis of pickup and delivery speeds amongst other metrics). Also, technical scaling involves the commonly understood notion of scaling, which is keeping the system afloat as the data and usage grow.

To scale our courier integrations, we have built a polymorphic integration model in our system. Such that the rest of the codebase is never aware of the specifics of a given courier. What it knows is that the courier has, for example, a create_shipment method that will return a tracking number like so:

class CourierX(Driver):
    def create_shipment(
        self,
        consignor,
        consignee,
        reference_number,
        weight,
    ) -> str:
    	pass

And when this courier can be passed wherever a Driver is expected.

def assign_shipment(
  shipment: Shipment,
  courier: Driver,
):
	tracking_number = courier.create_shipment(
    shipment.merchant,
    shipment.customer,
    shipment.reference_number,
    shipment.weight,
  )
  ...

With this approach, we can hide the complexity and variations in integrations in the driver and expose a unified interface. We have seen all sorts of craziness in courier APIs. From ancient SOAP APIs to inconsistent mess, you name it (sometimes a mix of both). For example, one courier had more than 30 required properties that needed to be sent. Only 10 of them are actually needed. The rest were static values that they never use. Yet, we have to send them. Other couriers have poor validations, so when you send an invalid request you get an Internal Server Error instead of a Bad Request with error messages.

One approach we take to ensure stability with scale is to push our couriers to adopt our S.O.P as their primary procedure and recommend using an off-the-shelf system to manage their operations instead of implementing their own. This last point is crucial because if they built their own, we would have to go through integrations and debugging problems specific to that one courier – which can delay their kickoff with us. However, if they used an off-the-shelf system, they would not have to reinvent the wheel and figure out how to map their ever-changing operations to technical requirements and fix API issues, and so on.

With this combination of practices, we have integrated more than 15 couriers natively (we ditched the off-the-shelf system we bought) in two months.

What's next? The Second 100K and Beyond

We believe that this is just the tip of the iceberg of what ZidShip's model can accomplish. We are working on expanding our service levels operations to new origins and expanding our coverage to new destinations, including international ones.

As of publishing this post, August 2021, ZidShip has already processed more than 200K shipments. We will continue to invest in improving the first-mile experience for the merchant and the couriers as it is the area where the most improvement can be made on our end. Improving the first-mile experience can significantly improve delivery times and decrease cost. For example, we will enhance our assigning algorithm to give cluster shipments to minimize courier trips. In addition, we will increase the coverage of the first-mile pickup to allow more merchants to get the whole ZidShip experience.