Lyearn is pleased to announce the open-sourcing of our robust and intuitive MongoDB ODM module for Go: mgod!
Simply put, mgod is:
Go module designed to improve the way developers interact with MongoDB in Go applications. It provides a structured way to map Go models to MongoDB collections, simplifying database interactions and ensuring error-free data handling in a wide range of applications.
Sounds interesting? Continue reading!
The challenges we faced
In our journey to develop mgod, we faced a series of significant challenges that highlighted the limitations of existing open-sourced Go modules. The most pressing issues included:
- Inadequate Schema Management: Existing solutions needed a more dynamic and efficient way to handle complex MongoDB schemas within Go applications. This often led to cumbersome workarounds and increased development time.
- Lack of Customization: We needed a solution that could be easily tailored to meet the specific needs of different projects, which was not adequately addressed by existing modules. A significant limitation we encountered was the rigid structure of these modules, which often forced the addition of fields that were not always necessary.
- Limited Validation Capabilities: We found that many modules provided only basic validation features, which were insufficient for ensuring data integrity in more sophisticated applications.
- Missing Multi-Tenancy Support: A major gap we observed was the absence of multi-tenancy support in existing modules, which limited their usability in scenarios requiring isolated data environments for different tenants.
The checklist for a better ODM
When conceptualizing, we focused on several key features that we wanted as built-in support in an ODM:
- One Schema for All:
mgodallows developers to reuse existing Go structs to define models and perform MongoDB operations. It is designed to handle complex document structures with ease, allowing developers to manage complex schemas as a single-type declaration only.
An example type declaration to show the ease of creating an ODM for User type using mgod .
- Customizable Options for Tailored Solutions: Recognizing the diverse needs of different data models,
mgodoffers extensive customization through field options, schema options, and meta fields, enabling developers to tailor the module to their specific requirements. For a detailed explanation of each option, check out the Features section inmgoddocumentation. - Advanced Data Validation Techniques: Ensuring data integrity is paramount, and
mgodachieves this with comprehensive validation mechanisms that go beyond the basics.
An example of data validation is as follows -
The above insertion will fail because of multiple validation errors -
IDis expected to be of primitive ObjectId type because ofmgoType:"id"the struct tag.Namefield is missing from the input doc but is required by default unlessbson:",omitempty”struct tag is declared against the field in the type declaration.
- Built-in Support for Multi-Tenant Systems: The module comes with built-in support for multi-tenant systems, making it ideal for applications requiring isolated data environments for different tenants. For a detailed walkthrough, check out Multi-Tenancy documentation.
- Support for Union Types:
mgodsupports union types, significantly expanding the capabilities for handling diverse data structures within applications. For examples and detailed explanations, check out Union Types documentation.
The challenges we faced
mgod is a Go module tailored for MongoDB to simplify the developer experience with database operations. It is a MongoDB ODM (Object Document Mapper) that allows for a structured and cohesive way to map Go models to MongoDB collections. The creation of mgod was driven by a need to consolidate MongoDB interactions within a single, unified schema in Go, a process that was previously fragmented and prone to errors.
This module sets itself apart by leveraging the strengths of Go’s type system and MongoDB’s flexibility. mgod facilitates an integrated approach to database operations, reducing redundant code and increasing type safety. Its design not only addresses the common pain points faced when using MongoDB with Go but does so in a manner that scales efficiently for production use. Developed initially for Lyearn's internal use, mgod has proven to be generic and versatile enough for any developer looking to harness the power of MongoDB within Go applications. The fact that mgod is used at scale in production environments is a testament to its stability and performance.
Architecture
The architecture of mgod is inspired by the easy interface of MongoDB handling using Mongoose and Typegoose libraries available in Node. Central to its design is the creation of a schema tree from GoLang type declaration, which acts as a blueprint for all interactions with the MongoDB database.
This structure allows for intelligent data validation, where every document is checked in detail against the schema to ensure accuracy and consistency. mgod stands out in its ability to dynamically manipulate data. Based on the schema configuration, the module can add, modify, or remove fields in real-time, offering developers a high degree of control and flexibility.
For example,
The above image illustrates the schema tree generated by mgod for a given Go struct. This tree acts as a conceptual map, outlining how each field in the Go struct corresponds to a node within the tree. For instance, the User struct is broken down into individual nodes such as _id, name, age, and meta, each representing a field in the MongoDB document. Substructures, like Metadata, are also represented, showing their nested relationship and how they connect back to the main tree. Nodes with dotted outlines are meta fields that will be added based on the schema options configuration while creating a new EntityMongoModel using mgod .
The above image depicts the transformation process from a Go struct instance to a BSON document, ready for storage in MongoDB. It provides a clear example of how mgod handles the conversion, including the handling of embedded fields and arrays. The illustration demonstrates the assignment of MongoDB-specific identifiers, like _id, and the preservation of data types across the translation from Go to BSON.
The above example exemplifies mgod ability to maintain data integrity and structure fidelity throughout the process, from application memory to database storage.
Getting Started
mgod is now open-sourced and available on Lyearn's GitHub site. For detailed documentation and a setup guide, visit mgod website.
Interested in Contributing?
We welcome contributions from the community! If you’re interested in improving mgod, please refer to our Contributing Guidelines on GitHub. We will be more than happy to discuss your suggestions!
