[root@bern sqoop]# mysql -uroot -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 123 Server version: 5.5.65-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> create database harsh8; Query OK, 1 row affected (0.05 sec) MariaDB [(none)]> use harsh8; Database changed
Pre-create the table to receive the datasets
MariaDB [harsh8]> CREATE TABLE staff ( id INT NOT NULL PRIMARY KEY, Name VARCHAR(20), Position VARCHAR(20),Salary INT,Department VARCHAR(10)); Query OK, 0 rows affected (0.26 sec)
MariaDB [harsh8]> show tables; +------------------+ | Tables_in_harsh8 | +------------------+ | staff | +------------------+ 1 row in set (0.00 sec)
Check the empty staff table structure
MariaDB [harsh8]> describe staff; +------------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +------------+-------------+------+-----+---------+-------+ | id | int(11) | NO | PRI | NULL | | | Name | varchar(20) | YES | | NULL | | | Position | varchar(20) | YES | | NULL | | | Salary | int(11) | YES | | NULL | | | Department | varchar(10) | YES | | NULL | | +------------+-------------+------+-----+---------+-------+ 5 rows in set (0.14 sec)
The empty table before the export
MariaDB [harsh8]> select * from staff; Empty set (0.00 sec)
Run the export to import the HDFS data into the hasrh8.staff table
21/06/29 10:23:05 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7.3.1.4.0-315 21/06/29 10:23:06 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 21/06/29 10:23:06 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 21/06/29 10:23:06 INFO tool.CodeGenTool: Beginning code generation 21/06/29 10:23:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `staff` AS t LIMIT 1 21/06/29 10:23:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `staff` AS t LIMIT 1 21/06/29 10:23:09 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/3.1.4.0-315/hadoop-mapreduce 21/06/29 10:23:22 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hdfs/compile/5097d7a0a163272ca680207ac06da7d5/staff.jar 21/06/29 10:23:22 INFO mapreduce.ExportJobBase: Beginning export of staff 21/06/29 10:25:10 INFO client.RMProxy: Connecting to ResourceManager at bern.swiss.ch/192.168.0.139:8050 21/06/29 10:25:13 INFO client.AHSProxy: Connecting to Application History server at bern.swiss.ch/192.168.0.139:10200 21/06/29 10:25:17 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /user/hdfs/.staging/job_1624952474858_0001 21/06/29 10:25:50 INFO input.FileInputFormat: Total input files to process : 1 21/06/29 10:25:50 INFO input.FileInputFormat: Total input files to process : 1 21/06/29 10:25:52 INFO mapreduce.JobSubmitter: number of splits:4 21/06/29 10:25:57 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1624952474858_0001 21/06/29 10:25:57 INFO mapreduce.JobSubmitter: Executing with tokens: [] 21/06/29 10:25:59 INFO conf.Configuration: found resource resource-types.xml at file:/etc/hadoop/3.1.4.0-315/0/resource-types.xml 21/06/29 10:26:01 INFO impl.YarnClientImpl: Submitted application application_1624952474858_0001 21/06/29 10:26:01 INFO mapreduce.Job: The url to track the job: http://bern.swiss.ch:8088/proxy/application_1624952474858_0001/ 21/06/29 10:26:01 INFO mapreduce.Job: Running job: job_1624952474858_0001 21/06/29 10:29:36 INFO mapreduce.Job: Job job_1624952474858_0001 running in uber mode : false 21/06/29 10:29:36 INFO mapreduce.Job: map 0% reduce 0% 21/06/29 10:33:15 INFO mapreduce.Job: map 75% reduce 0% 21/06/29 10:33:16 INFO mapreduce.Job: map 100% reduce 0%
YARN UI showing the job running and completing
Completed successfully
Data now uploaded in the destination table
MariaDB [harsh8]> select * from staff;
+-----+------------+-------------------+--------+------------+ | id | Name | Position | Salary | Department | +-----+------------+-------------------+--------+------------+ | 100 | Geoffrey | manager | 50000 | Admin | | 101 | Thomas | Oracle Consultant | 15000 | IT | | 102 | Biden | Project Manager | 28000 | PM | | 103 | Carmicheal | Bigdata developer | 30000 | BDS | | 104 | Johnson | Treasurer | 21000 | Accounts | | 105 | Gerald | Director | 30000 | Management | +-----+------------+-------------------+--------+------------+ 6 rows in set (0.28 sec)
Thank you for your response, but I asked a different question that can Sqoop export command be used to create a table in RDBMS. Apologies if my question was not clear before.