Greetings! This is the first post of hopefully many more to come related to a hobby project of mine - to develop an open source warehouse execution system (WES). I have wanted to undertake this project for several years, but only recently had the opportunity to start working on it.

Over the next several months, I plan on documenting the progress, designs, and decisions I made throughout the process. This is more of a hobby project for now that I am embarking, but it is probably one of the largest hobby projects I’ve ever taken on - both in scope/scale as well as length of time. In the end, if this paradigm and platform offers value for users in the logistics and warehousing industry, then all the better!

This is the first post where I’ll outline what a WES is as well as the goals of the entire project. It is a long read - I can already tell that from the onset of starting to write this. Feel free to skim/scroll to pertinent sections, but if you do read the whole thing - thank you in advance!

What are the goals?

The goals are very simple:

  • Create the WES. Implement an open source Warehouse Execution System with enough functionality that a small –> medium sized warehouse could utilize this in their day-to-day operations.
  • Document the process. Software development is convuluted. It’s hard to get right the first time. One of my goals is to document the process I went through to develop a software project at this scale. What tools were involved, architecture, design techniques, etc.
  • Utilize and learn new technologies and practices. in both the warehousing / logistics market as well as in the technical / software development realm.
  • Have fun. Most importantly, I want this project to be a great way for me to have fun as I build something I have a passionate interest in.
  • Profit? It doesn’t work that easily. But, if this does turn into a full time startup / gig, then so be it. That’s not the initial / foremost goal though.

A more altruistic goal of having an open source warehouse execution system might be more along the following lines:

Reduce the barriers of entry and enable people to build and deploy warehouse execution systems faster and more successfully. Thus, unleashing the potential of anyone to automate and fine tune their warehouse.

With that said, let’s dive in and take a closer look - starting with defining a WES.

What is a Warehouse Execution System?

In order to dive into the details, I think it’s important to start with defining a Warehouse Execution System - or WES for short. Thankfully, this is not a new concept to the world and we have plenty of definitions available to us. Let’s start off with a look at Wikipedia.

A WES is a computerized system used in warehouses and distribution centers to manage and orchestrate the physical flow of products from receiving through shipping

Let’s break this down a bit…

  • Computerized system. It is software and hardware working together to perform an action. Albeit, not as simple as this static web page, but also not as complex as ChatGPT.
  • Warehouse & Distribution Centers. These are where products are stored while they await going to customers or retail stores. Within logistics, it allows companies to hold just enough inventory to potentially fulfill customers for orders within the next 3-5 days. These vary in size - anywhere from a few thousand square feet to over a million square feet.
  • Manage/orchestrate physical flow of products. This is the nuts and bolts. It’s the brains of the warehouse and decides how the orders are fulfilled and how inventory is received into the system.

If you have ever ordered something online, you have in some way interacted with a warehouse and its corresponding warehouse execution system. This process may look something like the following:

Placing an order

  1. Customer places an order with a company.
  2. The order and associated details are stored in the company’s “Order Management System” (OMS)
  3. The OMS notifies the corresponding warehouse about the order via the WES
  4. The WES, at times in coordination with the Warehouse Management System (WMS), determines the best strategy to get the order fulfilled in the warehouse
  5. The order is fulfilled by fulfillment associates in the warehouse using the WES
  6. The order is shipped out of the warehouse to your house

Naturally, there are many more complexities and steps in the process. However, that is the high level steps that occur when you place an order and it gets to your doorstep. The goal of a WES is to ensure steps 3-5 are performed as efficiently as possible within a warehouse. There is a lot of history towards warehouse execution systems and how they came about to bridge the gaps between warehouse management systems and warehouse control systems. In some ways, a WES is simply the intersection of a Venn diagram between a WMS and a WCS. I’ll cover the history, concepts, and more in a future post. The gist for now is that there is an overlying software application which orchestrates the automation in the warehouse - this is the WES.

The goal of an open source, free WES is to provide anyone access to automating their warehouse through software. It is expensive to deploy WES software. While there are free and even some open source WES in the wild, most of the software is proprietary and does not allow the end user to modify or customize the solution themselves. The goal of this project is to provide a free, community driven WES. There is still a lot of growth in this industry and I feel there is at least room for one more - if only as a fun research experiment to see how far we can get with this project.

Let’s now dive into the topics of what we are building as part of this WES.

Core Functionality

The project will eventually provide basic functionality for nearly every area of a warehouse. My goal is to have these areas developed as pluggable modules that communicate with each other through a standard interface. This allows for other modules, in the future, to plug into the system and work without needing to change any of the implementation of other services/modules. These modules may have additional business rules and capabilities for plugins allowing users to customize the functionality to meet their particular business needs. For example, we may have different strategies for receiving and putaway of inventory that ensures certain products are putaway or processed in different areas than others.

The current plan is to split the core functionality into the following modules:

  • Inventory Management
  • Receiving
  • Putaway
  • Replenishment
  • Cycle Counting
  • Order Management
  • Picking / fulfilling orders
  • Consolidating orders
  • Packing orders
  • Audit / Quality control
  • Loading / Shipping orders
  • Basic sortation / routing
  • Returns

While these modules may change over time - the goal is to at minimum, provide this core functionality. For the initial rollout, my goal is to focus on automating more of the manual activities in a warehouse. Currently, I don’t have plans in the immediate future to include core functionality for Automatic Storage / Retrieval Systems or other robotic implementations in warehouses. I have thought a little towards how to support these in the future, and similar to the rest of the functionality, these will more than likely build out as individual module(s). My intent is to start “small” with a core set of features and then evolve from there.

Quick side note: Currently, I’m tracking all of my progress and requirements within an on-prem instance of Jira. While this on-prem environment may always be how I handle my detailed requirements, I plan to have a more, larger picture, board that is accessible to the public. I may end up utilizing something as simple as GitHub Issues for this piece.

Ancillary Functionality

There are also several ancillary modules to support the core functionality. While these modules may not directly drive warehouse activities, they are still required to manage users, labor, host communications, and event tracking. Below is an initial list of these supporting modules.

  • Historical, event driven logging
  • User, role, access management
  • A full working UI to support all of the main activities within the warehouse
  • Labor management
  • Host management / communication
  • Interface / communication mechanisms

Artificial Intelligence / Machine Learning

One item not listed in any of the above modules that is gaining a lot of momentum and traction in the warehousing space and throughout most of software development is the focus on artificial intelligence and machine learning. While I have some ideas around this, especially in the labor planning and order/inventory management side, I will instead write a more detailed post in the future.

Technology / Architecture

I will spend a lot more time talking through the technology, architecture, design, and overall engineering of this project throughout several future posts. I’ll stick to just highlighting the very minimal basics to give people the idea of how this is built.

When choosing an architecture and tech stack for this project, I wanted to get somewhere that allowed for as much flexibility and modularity as possible while still keeping the project somewhat simple to deploy and maintain over time. While a lot of companies are moving to a version of microservices, I did not want a pure microservice approach. I also didn’t want to go down the path of having a monolith for the application either. I wanted to settle somewhere between these two spectrums. I’d argue the architecture for this is more based on modules / modular services created based on Domain Driven Design (DDD). I like it so far, but there are some improvements I still want to make.

The project is currently comprised of the following technologies:

  • C# / ASP.NET 6 - Backend Services
  • Postgres / MongoDB - Data storage. Postgres is used for real time data and the plan is to use MongoDB for historical data.
  • Vue.js - UI
  • Python - Simulation / End-to-End testing, as well as future AI/ML training.
  • RabbitMQ - message queues / topics - most of the communication for this project is using commands / events.
  • Docker - used for container building / orchestration.
  • Azure - used to host the entire application.

In the future, I may start to research and adapt more technologies - both for the fun of it, as well as to improve the overall architecture, structure, or performance of the project.

There are currently three main subprojects:

  • Service - this is where all of the backend services/modules reside. They are, currently, each standalone projects that could be spun up individually if needed and work with other outside services / systems.
  • UI - this is the UI / visibility into the system. It is primarily written in Vue.js and provides the dashboards, reports, and other helping screens that operators, supervisors, and other staff may use to monitor the progress of the warehouse throughout the day.
  • Integration - this is currently the subproject that houses all of the deployments and system level testing. It’s called integration, because this is the glue code that brings everythign together, deploys it, and ultimately tests the end-to-end functionality.

What’s next?

Over the next several days, weeks, and months, I plan to write and post on the progress of this project. You should expect posts to generally fall into one of the following broad topics:

  • Engineering - these posts focus on the overall architecture, tech stack, and other engineering or technical processes I went through in making this project.
  • Product and Design - in these posts, I’ll focus on the roadmaps, strategy, and vision of the product. Coming from a design aspect, these posts will discuss the process I went through in designing the UX of the system and any other pertinent design concepts/details.
  • Development Process - there are tons of processes out there to help improve product and software development. I’ll talk through some of my processes and overall industry best practices and standards as it revolves around a process oriented project.
  • Demos - As I release and update demos, I’ll share information on what’s new, how to use the demos, and what’s planned for future demos. This will also include a link to the open source project once I port everything over to a new public repository.
  • Data / AI / ML - these posts focus on the data, machine learning, and AI concepts developed in this project. The first of these will entail some of my plans and vision in this area - as it is still fairly new and growing, especially in the warehousing / logistics industry.

And, the most important aspect, the actual open source aspect to this. Over time, I plan to create and then add code to the open source repository. While I have started a little bit on this project, I haven’t made any of the source code public yet primarily because I am still figuring out how I want to go about that process. Either release by module until everything is out there or some other means.

Conclusion

That was a long writeup - and if you stuck through it this long, thank you. This is definitely a fun project and I’m looking forward to writing more on the process, decisions, and direction. As of now, if you want to reach out to me, you may email me, find me on Twitter, or GitHub.