The Spark Universal Migrator is a data migration Spark application that fetches data from an Oracle database and stores it in a Hive database. It utilizes JDBC to connect to Oracle, processes and converts the data into a Hive-compatible schema, and then transfers it to Hive. The migration process employs iterators to generate queries for data retrieval and utilizes a BlockingQueue in conjunction with a ThreadPoolExecutor for parallel processing of data. Additionally, the application incorporates an Oracle ...
| License | Apache 2.0 |
|---|---|
| Tags | github |
| Organization | queukat |
| HomePage | https://github.com/queukat/spark_oracle_hive_streaming 🔍 Inspect URL |
| Date | May 13, 2023 |
| Files | pom (2 KB) jar (26 KB) View All |
| Repositories | Central |
| Ranking | #580496 in MvnRepository (See Top Artifacts) |
| Scala Target | Scala 2.12 (View all targets) |
| Vulnerabilities | Vulnerabilities from dependencies: CVE-2023-22946 |
Compile Dependencies (5)
| Category/License | Group / Artifact | Version | Updates | |
|---|---|---|---|---|
| JDBC Driver | com.oracle.database.jdbc » ojdbc8 | 21.6.0.0.1 | 19.29.0.0 | |
| Dist Computing Apache 2.0 | org.apache.spark » spark-core_2.121 vulnerability | 3.0.0 | 4.0.1 | |
| SQL Lib Apache 2.0 | org.apache.spark » spark-sql_2.12 | 3.0.0 | 4.0.1 | |
| Hadoop Query Engine Apache 2.0 | org.apache.spark » spark-hive_2.12 | 3.0.0 | 4.0.1 | |
| JVM Languages Apache 2.0 | org.scala-lang » scala-library | 2.12.11 | 3.7.4 |
Licenses
| License | URL |
|---|---|
| Apache 2 | http://www.apache.org/licenses/LICENSE-2.0.txt |
Developers
| Name | Dev Id | Roles | Organization | |
|---|---|---|---|---|
| yaroslav | queukat<at>gmail.com | queukat |