We are planning to use Iceberg tables instead of PostgreSQL. For PostgreSQL, we were using Flyway for database migrations. So, I wonder if it is possible to do it for Iceberg tables, as well. ChatGPT said it is possible through Athena but I couldn't make it work. Anyone has any experience or suggestion for that?
I saw pyiceberg but it is not exactly a database migration tool even though it might be handy to use that one. /
We are planning to use Iceberg tables instead of PostgreSQL. For PostgreSQL, we were using Flyway for database migrations. So, I wonder if it is possible to do it for Iceberg tables, as well. ChatGPT said it is possible through Athena but I couldn't make it work. Anyone has any experience or suggestion for that?
I saw pyiceberg but it is not exactly a database migration tool even though it might be handy to use that one. https://py.iceberg.apache.org/
I think this blog is helpful for your use case. It use Amazon Data Firehose to help you maintain fresh, analytics-ready data in data lake without implementing complex ETL processes or managing infrastructure.
Near Real-Time Database Replication to Apache Iceberg Table on S3 Using Amazon Data Firehose
