Cost-Efficient and GDPR-Compliant: Datopian Migrates Open Data Denmark from Google Cloud to Hetzner

Context

Why did the Danish Government choose Datopian and CKAN over Big Tech for hosting Open Data Denmark on European soil? The answer lies in the need for enhanced data sovereignty, strict GDPR compliance, and the rising costs of non-European cloud providers.

In 2024, Open Data Denmark, a national portal that hosts public datasets, fostering transparency and innovation through open data initiatives, sought a reliable partner for hosting, support, and maintenance services. With the Danish government’s focus on data sovereignty, seamless data accessibility, enhanced security, and cost-efficiency, they turned to Datopian - an industry-leading provider of CKAN and open-source data solutions - to migrate the platform from Google Cloud Platform (GCP) to Hetzner’s European-based data centers.

The Challenge

The migration from Google Cloud Platform to Hetzner was driven by growing concerns over trust, data privacy and data sovereignty issues related to GCP. In Denmark, Google's brand suffered a setback when the Danish Data Protection Agency (DPA) imposed a ban on the use of Google Workspace in the Elsinore municipality, citing data privacy concerns about the safety of Google’s European infrastructure. Although Google Workspace and GCP are separate products, the incident raised doubts about Google's overall trustworthiness as a reliable provider for Danish governmental projects.

This situation prompted the Danish government to seek a more reliable, European-based alternative that aligned with their values of data sovereignty and GDPR compliance, ultimately leading to the partnership with Datopian and the migration to Hetzner's Germany-based data centers. You can also find more about the case here - Datatilsynet (Denmark) - 2023-431-0001 - GDPRhub.

Initially, the Open Data Denmark portal was maintained by the Datopian team and hosted on Google Cloud Platform using data centers located in Europe. While GCP offered robust infrastructure, Denmark’s commitment to data sovereignty and cost-efficiency required a new strategy. Datopian was tasked with:

• Migrating the portal’s hosting infrastructure from GCP’s European data centers to Hetzner's data centers in Germany.
• Providing comprehensive support and maintenance services to ensure high availability, performance, and security of the portal. This included services such as CKAN’s Python application, PostgreSQL databases, Apache Solr search engine, Redis, and more.
• Reducing hosting costs while maintaining or enhancing service quality.
• Ensuring compliance with EU data protection regulations by utilizing Hetzner’s German-based data centers.

The Solution

Migration to Hetzner

In the summer of 2024, Datopian successfully migrated the hosting infrastructure for Open Data Denmark from GCP to Hetzner's data centers in Germany. The choice of Hetzner was driven by three key factors:

• Cost-efficiency: Hetzner’s competitive pricing model significantly reduced operational costs. Compared to the rising costs of GCP, Hetzner provided a more sustainable solution, cutting overall costs by over 30%, ensuring the long-term financial sustainability of the portal’s operations.
• Data sovereignty: Hosting within Germany with Hetzner ensured full compliance with EU regulations, particularly the General Data Protection Regulation (GDPR). This was a key priority for the Danish government, which required a solution that prioritized data privacy and sovereignty.
• Performance: Hetzner’s robust hardware and network infrastructure provided superior performance, including lower latency and faster access for users interacting with the Open Data Denmark portal. This performance boost increased overall portal usage and improved the end-user experience.

The migration process was carefully planned and executed by Datopian’s technical team to ensure seamless data migration with minimal downtime. The process involved:

• A comprehensive audit of the existing GCP infrastructure.
• Setting up a parallel environment on Hetzner to mitigate risks.
• Implementing a phased migration strategy that included data replication, extensive testing, and a final switchover to ensure a smooth transition. This approach minimized disruption to portal services while ensuring complete data integrity.

Hosting, Support, and Maintenance

After the migration, Datopian took on the responsibility of providing comprehensive hosting, support, and maintenance services. This included:

• Hosting Management: Leveraging Hetzner's dedicated servers, Datopian managed all aspects of hosting, including server configuration, resource optimization, and monitoring.
• Proactive Support: Datopian established a 24/7 support structure to promptly address any technical issues, ensuring the portal's uninterrupted operation. Automated monitoring tools were employed to detect and resolve issues before they impacted the user experience. This proactive approach guaranteed the portal's high availability and uninterrupted operation.
• Security Enhancements: The migration provided an opportunity to enhance the portal's security. Datopian implemented rigorous security measures, including regular vulnerability assessments, automated backups, and disaster recovery plans. These measures ensured that Open Data Denmark complied with the highest security standards, safeguarding sensitive public data.
• Ongoing Maintenance: Regular updates to the portal's software and infrastructure were carried out to ensure compliance with the latest security standards and performance improvements. Regular updates were made to CKAN, PostgreSQL databases, and other core components to keep the system running efficiently.

Results

The migration to Hetzner's data centers and Datopian’s ongoing support produced several tangible benefits for Open Data Denmark:

• Cost Savings: Hosting, maintenance, and support costs were reduced by over 30% compared to GCP, freeing up financial resources that could be reinvested in open data initiatives and other strategic projects.

• Enhanced Performance: Users experienced improved performance with reduced latency and faster data access, leading to increased portal usage and engagement.

• Improved Data Sovereignty: Hosting in Germany provided a higher level of data sovereignty, addressing privacy concerns and ensuring compliance with EU regulations.

• Increased Reliability: With Datopian’s proactive support and robust maintenance practices, the portal maintained high availability and experienced minimal downtime.

Conclusion

By partnering with Datopian for hosting, support, and maintenance, Open Data Denmark successfully migrated to a more cost-effective, high-performance, and secure hosting solution. This case study underscores Datopian's expertise in delivering tailored open-source solutions, helping organizations like Open Data Denmark harness the full potential of their data while maintaining compliance and performance.

In addition to this migration project, Datopian has a long-standing partnership with Open Data Denmark (ODDK). Since August 2018, we’ve collaborated with ODDK to modernize and upgrade their CKAN classic portal. This involved improving the portal’s user experience, enhancing security measures, and integrating multilingual support for Danish, English, and French.

For organizations seeking a reliable partner to enhance their open data platforms, streamline cloud infrastructure, or ensure compliance with EU regulations, Datopian provides a proven track record of success.

If you're looking for a trusted partner to elevate your data management capabilities, we invite you to explore how Datopian can tailor solutions to meet your unique needs. Reach out to us to discuss how we can help you achieve your goals while maintaining the highest standards of performance and compliance.