Description Repo Name: cml_vscode_with_python310 Repo Info: VsCode using Python 3.10, instead of the existing VsCode example using Python 3.7.  Repo URL: https://github.com/ryancicak/cml_vscode_with_python310   ... View more

I recently received a question "We'd like to have P ython and R installed in the docker image for a package.  Is it possible to have such customized docker with python >=3.7, R>=4.0 and CUDA==11.7?"   I'd like to go through how I accomplished this using the pbj-workbench-r4.1-standard.Dockerfile.  This runtime has the essentials (Python and R) except for CUDA (NVIDIA GPU).  Therefore, we'll use pbj-workbench-r4.1-standard.Dockerfile as a base image.  It's important to note that CML requires a runtime kernel that needs to be either R or Python, it can't be both.  I'll go through three different iterations of this image:   Iteration 1 - Since this image already has R and Python installed, we'll change the runtime NOT to use R ryancicak/pbj_r-and-python:v9 -> Python 3.10 (GitHub Tag: python/3.10/standard_dockerhub_v9) https://github.com/ryancicak/pbj_r-and-python/blob/python/3.10/standard_dockerhub_v9/Dockerfile   Iteration 2 - Go back to using R (as we did initially before removing R as the runtime kernel) in Iteration 1 ryancicak/pbj_r-and-python:v10 -> R 4.1 (GitHub Tag: r/4.1/standard_dockerhub_v10) https://github.com/ryancicak/pbj_r-and-python/blob/r/4.1/standard_dockerhub_v10/Dockerfile   Iteration 3 - Searching the nvidia/cuda dockerhub tags, I found version 11.7.1, with similarities to  https://github.com/cloudera/ml-runtimes/blob/public-runtimes/pbj-workbench-python3.10-cuda.Dockerfile :   # Copyright 2022 Cloudera. All Rights Reserved. FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu20.04 RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub    to   # Copyright 2022 Cloudera. All Rights Reserved. FROM nvidia/cuda:11.7.1-cudnn8-devel-ubuntu20.04 RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub   *Notice "-cudnn8-devel-ubuntu20.04" is identical, as well as the following "RUN" line *Validating cuda 11.7 is running   If we compare the changes between iteration 1 and 2, it's ever so slight when comparing tags on the Dockerfile.  Again, we're only changing the runtime kernel since the image has both R and Python installed.  Feel free to try these different iterations in the Docker Hub tags as specified within CML by adding these as new Runtimes:   Iteration 1 -  ryancicak/pbj_r-and-python:v9 Iteration 2 - ryancicak/pbj_r-and-python:v10 Iteration 3 - ryancicak/pbj_r-and-python:v11   While the versioning is different in CML, they're all basically using the same docker image as asked in the original question above.     Shout-out to @amarinovszki for his collaboration!  Thanks Árpád!   ... View more

Have you stumbled upon existing articles on only reading from a managed Hive table using R within CML, but you really want to read/write to an external table (not a Hive Managed table)?  Then you've come to the right place!     There are complexities to reading/writing to a Hive-managed table since ACID is involved, as the HWC piece solves.  In 2023, if you're looking for ACID capabilities then using Iceberg table format instead of Hive-managed tables is recommended anyways. The Iceberg table format is fully supported throughout CDP.     Let's get on with the steps to easily read/write using R to an external table within CML!  Again, without the need for HWC.     Step 1: C reated two tables (external) and inserted records *In this case, I used Cloudera Data Warehouse with an Impala Virtual Warehouse   create database cloudera_r; create external table cloudera_r.people_table (id int, name string) location 's3a://ryancicak/people_table'; insert into cloudera_r.people_table values (0,'ryan'), (1, 'paul'), (2, 'oleksandr'), (3, 'oliver'), (4,'jd'); create external table cloudera_r.people_new_table (id int, name string) location 's3a://ryancicak/people_new_table'; insert into cloudera_r.people_new_table values (5,'danielle'), (6,'nilesh');   Step 2: Modifying the code in the article above (stripping out HWC) - I'm able to run the code (which I'll include a snippet below so you can copy) Since the Hive Metastore contains the S3 location from Impala in Step 1, we do not need to specify the S3 location for Spark to read/write. We need the spark.yarn.access.hadoopFileSystem to contain the S3 location that our table is - so Spark has access to read/write.  If you make a modification on spark.yarn.access.hadoopFileSystem - it's best to start a new session.   #CONNECT TO DATA LAKE USING SPARK #Run this once #install.packages("sparklyr") #Add library library(sparklyr) library(dplyr) library(DBI) #Add spark_config() spark_config() config <- spark_config() #Add configuration config$spark.security.credentials.hiveserver2.enabled="false" config$spark.datasource.hive.warehouse.read.via.llap="false" config$spark.sql.hive.hwc.execution.mode="spark" config$spark.yarn.access.hadoopFileSystems="s3a://ryancicak/" #config$spark.datasource.hive.warehouse.read.jdbc.mode="spark" #Connect with spark sc <- spark_connect(config = config) #Change database if necessary spark_session(sc) %>% invoke("sql", "USE cloudera_r") peopleframe <- tbl(sc, sql("select * from cloudera_r.people_new_table")) print(peopleframe) #spark_write_csv(peopleframe,"s3a://ryancicak/testing_people.csv", header=TRUE) spark_write_table(peopleframe, "cloudera_r.people_table", mode = "append") #Read and show table intDf1 <- sparklyr::spark_read_table(sc, 'people_table') sparklyr::sdf_collect(intDf1) peopleframe <- tbl(sc, sql("select * from cloudera_r.people_table")) print(peopleframe)   Notice that the first print table is showing the two rows, Danielle and Nilesh.  The second table print is showing the table people_table after we wrote the two new rows, where instead of five initial records, we show seven (since we appended). I do print the same table twice - just showing two different ways to read from a table (I prefer the SQL option). If you do not choose the correct bucket name (the S3 bucket where you're reading/writing tables) - you'll receive an error.  In my case, I receive a RAZ 401 error (since I have RAZ enabled).  If I change my bucket after running one in my session, I'll still get the error.  I need to stop and then start a new session to take in the correct bucket.  I can add multiple S3 locations separated with commas. Notice that I changed my hadoopFileSystems to "differents3bucket" and receive the access error.  Even if I change back to "ryancicak", I still receive an access error until I start a new session.     @pauldefusco added some more examples to his GitHub repo that may be helpful:  SparklyR_CML_Example.   Shout-out to @pauldefusco, @ozarate, @aakulov, and @jagadeesan for helping with these steps! ... View more

In a recent release, CDW is now fully integrated with RAZ.  This means once you provision an environment in CDP with RAZ enabled, adding an S3 bucket into CDW follows the same principles as the services within CDP (CML, CDE, COD, CDF, and DataHub).   I was able to add a new S3 bucket called "ryancicak" in two steps, after receiving the 403 Forbidden error within Hue: Two steps:   Step 1 - Since I used the -datalake-admin-role for RAZ, I modified the "datalake-admin-s3-policy to include my new bucket "ryancicak" with the two lines: "arn:aws:s3:::ryancicak", "arn:aws:s3:::ryancicak/*"   *Notice I also removed the subdirectory from my cicak-cdp-stack to include the root directory.  This is important to make this modification as well.  Instead of "cicak-cdp-stack/my-data", I state "cicak-cdp-stack" Step 2 - I added a new policy in Ranger under cm_s3 for my bucket "ryancicak", for the users rcicak (me), and Hive.     DONE - When re-running the create table using the bucket "ryancicak", no more errors!  The table "rupert" is now reading/writing to the "ryancicak" bucket.  Since I have my Ranger (RAZ) policy created, from step 2, I'm able to access this bucket from CDP (CML, CDE, COD, CDF, and DataHub).   Note: I used the default "-datalake-admin-role" that was created through the CloudFormation script in the AWS Quickstart.  In my case, I named the stack "cicak-cdp-stack" in the CloudFormation script.     That was EASY: ... View more

Special shout-out to @zoram, who provided these instructions.  This article will take you through these steps (executing each step with a screenshot).  This article specifies an edge case that involves processing large amounts of non-tabular datasets such as images.  The data lake is the preferred storage for CML as object storage can scale to billions of objects with tabular data.  If you qualify for this edge case (non-tabular datasets), please delete the dataset from EFS/NFS as soon as possible (after processing), since large amounts of data on EFS/NFS negatively affect backup and recovery times.   When it comes to non-tabular data, you may not want to use object storage with standard readers such as Boto3 (for AWS), which has more latency attached to each read.  If we move to a faster I/O such as EFS, which is already used by your CML workspace, we can load data to your existing CML workspace EFS.  It's a simple process that we'll take you through:   Step 1 -  When dealing with a large number of files, switch your CML workspace to provisioned throughput mode from the AWS EFS console.  Increase this to what you're willing to pay for (I/O) and keep it provisioned throughput until you're done processing the files in EFS.   Step 2 - Login to the CML UI, and launch a session in the project.  The project you'd like to process files within your workspace's EFS.     Step 3 - Within your session, launch the "Terminal Access"   Step 4 - Within your Terminal Session, run the following command and write down the output, as you'll need the output later.  You can stop your session (it's only for this step).   df -h | grep home           Step 5 - Using kubectl, you'll determine which node is running ds-vfs.  In order to access kubectl on your laptop, you'll need to download the Kubeconfig for the workspace you're accessing.     Run the following command:   kubectl get pods -n mlx -o wide | grep ds-vfs           (In case you need to download the Kubeconfig on the workspace):   Step 6 - In case you do not have a security policy in place to access SSH on the node you found in step 3, you'll want to add a security policy (to SSH into the node).  After you're done with these steps, feel free to remove this security policy (to SSH into the ds-vfs node.     Step 7 - SSH into the node from steps 3 & 4   Step 8 - Determine the EFS mount point, and then sudo su (so you're root moving forward)   cat /proc/mounts | grep projects-share | grep -v grafana sudo su           My output:   127.0.0.1:/ /var/lib/kubelet/pods/eb4b15b6-631a-428b-92df-e0d31074e7f9/volumes/kubernetes.io~csi/projects-share/mount nfs4 rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,noresvport,proto=tcp,port=20081,timeo=600,retrans=2,sec=sys,clientaddr=127.0.0.1,local_lock=none,addr=127.0.0.1 0 0 127.0.0.1:/ /var/lib/kubelet/pods/d3c5b66c-c84a-40cc-bed9-79fb95f3a7ad/volumes/kubernetes.io~csi/projects-share/mount nfs4 rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,noresvport,proto=tcp,port=20246,timeo=600,retrans=2,sec=sys,clientaddr=127.0.0.1,local_lock=none,addr=127.0.0.1 0 0 127.0.0.1:/ /var/lib/kubelet/pods/1ff3a121-e7fb-4475-974f-92446f65a773/volumes/kubernetes.io~csi/projects-share/mount nfs4 rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,noresvport,proto=tcp,port=20316,timeo=600,retrans=2,sec=sys,clientaddr=127.0.0.1,local_lock=none,addr=127.0.0.1 0 0 127.0.0.1:/ /var/lib/kubelet/pods/96980148-9630-4f5e-90b6-05970eacdf1f/volumes/kubernetes.io~csi/projects-share/mount nfs4 rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,noresvport,proto=tcp,port=20439,timeo=600,retrans=2,sec=sys,clientaddr=127.0.0.1,local_lock=none,addr=127.0.0.1 0 0             Step 9 - Go to the projects file system in my case (you'll pull the last piece from your output in step 3)   cd /var/lib/kubelet/pods/eb4b15b6-631a-428b-92df-e0d31074e7f9/volumes/kubernetes.io~csi/projects-share/mount/projects/0/1           Step 10 - Create a new folder that you'll use to load your files    mkdir images       Step 11 - Validate the directory permissions are set correctly.  Files and directories must be owned by 8536:8536   ls -l | grep images     output:   drwxr-sr-x 2 8536 8536 6144 Mar 13 21:01 images     IMPORTANT: If you're loading MANY files - it's beyond important to add this new directory to the gitignore - so these files don't get committed to GIT!   Step 12 - Copy your files into the new directory from step 7, from S3.  You'll likely want to figure out S3 authn/authz for the copy.  It would be a best practice to script this copy and run it with nohup as a background command.  That way the copy doesn't get terminated if the SSH session times out.     Step 13 - Your files are now available in the new directory.   IMPORTANT: It's general best practice to not load all your files in one directory.  Instead, create multiple subdirectories and load say 10k files per subdirectory.  Within your code (say Python), you can iterate subdirectories and process the files.  This way, you aren't overwhelming your code - loading all files at once!   Warning: Do the cost estimate on using EFS vs S3 - before going into sticker shock on using higher-cost storage within AWS.  Again - after you're done processing your data within EFS/NFS, please remember to delete this data  since large amounts of data on EFS/NFS negatively affect backup and recovery times.    ... View more

If you've ever wanted to automate CDP, the CDP CLI is the tool to use.  Anything from pulling Virtual Warehouse details to spinning up a new Virtual Warehouse, the CDP CLI is a powerful tool.  Not only Data Warehouse driven, the CDP CLI encompasses all pieces of CDP.     CML is an IDE for all things.  Working on a project from within CML should be a simple and straightforward process.  If you're creating an application that will interact with CDP in one way or another, the CDP CLI can be installed in less than sixty seconds.  Once installed within your CML Project, you'll have the full power of the CDP CLI at your fingertips.     Step 1: Within your session launch the terminal access    Step 2: Within your session Terminal, type pip install cdpcli and click enter   pip install cdpcli    Step 3: Generate an API Access Key (if you don't already have one) https://docs.cloudera.com/cdp-public-cloud/cloud/cli/topics/mc-cli-generating-an-api-access-key.html You'll receive a cdp_access_key_id and cdp_private_key Step 4: Within your session Terminal, type cdp configure   cdp configure    You'll input your CDP Access Key ID and CDP Private Key, you can leave the CDP Region [None] and CDP Endpoint URL [None] That's it!  Now you can run all the CDP CLI commands within your CML Project!  As I said, you're done in sixty seconds! ... View more

Throughout my seven years working with Cloudera/Hortonworks, I'm always learning new things.  One thing I've learned from the Cloudera/Hortonoworks merger was how amazing CDSW/CML is as a product.  CML isn't only for Data Scientists, it's for anyone that needs an IDE (Integrated Development Environment).  Coming from a development background working with Eclipse and IntelliJ, you become dependent on a solid IDE.   In the past, I've used IDEs to develop applications, and then a build process would eventually deploy to a run environment.  This is where CML shines, you're able to run your applications within CDP at scale with enormous amounts of data.  Anyone using CML is over the moon running their applications/projects within CML because of the simplicity, as I'll demonstrate below.  The tagline of CML is "BYOL" (Bring your own libraries), meaning ALL libraries are welcome (outside of the Cloudera ecosystem).  This differentiates Cloudera from others such as native Azure, where native Azure is HIGHLY dependent on all things Microsoft (unless the application owner created something native Azure-specific).  I'll demonstrate how easy deploying a third party such as Django "The web framework for perfectionists with deadlines." where the installation/run feels like you're running on your local laptop, instead of a highly scalable IDE that runs anywhere.     Remember that CML runs ANYWHERE, within the cloud providers such as Azure, AWS, or GCP, and on-premise.  Cloudera abstracts out the complexities.  Using CML, we'll go from installing Django and then running Django in a matter of minutes.     Step 1: Find the read-only URL to run your embedded application (Django)   import os url=os.environ["CDSW_ENGINE_ID"]+"."+os.environ["CDSW_DOMAIN"] print("http://read-only-%s"%url)     Step 2: Install Django   !pip install django     Step 3: Create a Django project as instructed here   !django-admin startproject mysite cd mysite     Step 4: Modify the settings.py file adding the 'read-only-%s' value from step 1 and localhost   ALLOWED_HOSTS = ['localhost','read-only-yourhostnamefromstep1']     Step 5: Run Django   !python manage.py runserver localhost:$CDSW_READONLY_PORT     That's IT!  If you'd like to access your Django page, navigate to the URL in step 1!  Nothing specialized for CML, as we say BYOL!   ... View more

In order to debug pairing DLM, you'll need the following pre-req: 1) Root access to the DPS VM Problem statement - have you received an error when pairing a cluster? Follow these step-by-step instructions to access the DLM log, to gain granular log information that will help you debug: 1) Run the "sudo docker ps" command to gain the container id for "dlm-app": In the image above, the container id for "dlm-app" is "83d879e9a45e". 2) Once you receive the container id, you can run the following command "sudo docker exec -it 83d879e9a45e /bin/tailf /usr/dlm-app/logs/application.log" This will give you insight into the DPS-DLM application, in the example above you'll see "ERROR". The error log will post once you click "pair" in the DLM UI. Using the information from the log, you'll be able to troubleshoot your issue. ... View more

Reading and writing files to a MapR cluster (version 6) is simple, using the standard PutFile or GetFile, utilizing the MapR NFS. If you've searched high and low on how to do this, you've likely read articles and GitHub projects specifying steps. I've tried these steps without success, meaning whats out there is too complicated or out-dated to solve NiFi reading/writing to MapR. You don't need to re-compile the HDFS processors with the MapR dependencies, just follow the steps below: 1) Install the MapR client on each NiFi node #Install syslinux (for rpm install) sudo yum install syslinux #Download the RPM for your OS http://package.mapr.com/releases/v6.0.0/redhat/ rpm -Uvh mapr-client-6.0.0.20171109191718.GA-1.x86_64.rpm #Configure the mapr client connecting with the cldb /opt/mapr/server/configure.sh -c -N ryancicak.com -C cicakmapr0.field.hortonworks.com:7222 -genkeys -secure #Once you have the same users/groups on your OS (as MapR), you will be able to use maprlogin password (allowing you to login with a Kerberos ticket) #Prove that you can access the MapR FS hadoop fs -ls / 2) Mount the MaprR FS on each NiFi node sudo mount -o hard,nolock cicakmapr0.field.hortonworks.com:/mapr /mapr *This will allow you to access the MapRFS on the mount point /mapr/yourclustername.com/location 3) Use the PutFile and GetFile processor referencing the /mapr directory on your NiFi nodes *Following 1-3 allows you to quickly read/write to MapR, using NiFi. ... View more

Step-by-step instructions on how-to install a CRAN package from a local repo - without an internet connection. I'll be installing the package called "tidyr". In order to fully install the package, I need to first download tidyr and all dependencies. To do this, I use the CRAN PACKAGE: https://cran.r-project.org/src/contrib/PACKAGES 1) Finding "Package: tidyr", I can see the dependencies (imports): Imports: dplyr (>= 0.7.0), glue, magrittr, purrr, Rcpp, rlang, stringi, tibble, tidyselect and from them, I can build a list of all the imported packages I need: assertthat_0.2.0.tar.gz BH_1.66.0-1.tar.gz bindr_0.1.1.tar.gz bindrcpp_0.2.tar.gz cli_1.0.0.tar.gz crayon_1.3.4.tar.gz dplyr_0.7.4.tar.gz glue_1.2.0.tar.gz magrittr_1.5.tar.gz pillar_1.2.1.tar.gz plogr_0.1-1.tar.gz purrr_0.2.4.tar.gz Rcpp_0.12.16.tar.gz rlang_0.2.0.tar.gz stringi_1.1.7.tar.gz tibble_1.4.2.tar.gz tidyr_0.8.0.tar.gz tidyselect_0.2.4.tar.gz utf8_1.1.3.tar.gz R6_2.2.2.tar.gz pkgconfig_2.0.1.tar.gz 2) Download the packages from https://cran.r-project.org/src/contrib/ 3) Create a directory to emulate the CRAN repo, in my example I created /tmp/ryantester/src/contrib 4) Create a PACKAGES file within /tmp/ryantester/src/contrib - since this tutorial covers tidyr, I'll include the necessary packages Package: R6 Version: 2.2.2 Depends: R (>= 3.0) Suggests: knitr, microbenchmark, pryr, testthat, ggplot2, scales License: MIT + file LICENSE NeedsCompilation: no Package: pkgconfig Version: 2.0.1 Imports: utils Suggests: covr, testthat, disposables (>= 1.0.3) License: MIT + file LICENSE NeedsCompilation: no Package: bindr Version: 0.1.1 Suggests: testthat License: MIT + file LICENSE NeedsCompilation: no Package: bindrcpp Version: 0.2 Imports: Rcpp, bindr LinkingTo: Rcpp, plogr Suggests: testthat License: MIT + file LICENSE NeedsCompilation: yes Package: plogr Version: 0.1-1 Suggests: Rcpp License: MIT + file LICENSE NeedsCompilation: no Package: BH Version: 1.66.0-1 License: BSL-1.0 NeedsCompilation: no Package: plogr Version: 0.1-1 Suggests: Rcpp License: MIT + file LICENSE NeedsCompilation: no Package: dplyr Version: 0.7.4 Depends: R (>= 3.1.2) Imports: assertthat, bindrcpp (>= 0.2), glue (>= 1.1.1), magrittr, methods, pkgconfig, rlang (>= 0.1.2), R6, Rcpp (>= 0.12.7), tibble (>= 1.3.1), utils LinkingTo: Rcpp (>= 0.12.0), BH (>= 1.58.0-1), bindrcpp, plogr Suggests: bit64, covr, dbplyr, dtplyr, DBI, ggplot2, hms, knitr, Lahman (>= 3.0-1), mgcv, microbenchmark, nycflights13, rmarkdown, RMySQL, RPostgreSQL, RSQLite, testthat, withr License: MIT + file LICENSE NeedsCompilation: yes Package: utf8 Version: 1.1.3 Depends: R (>= 2.10) Suggests: corpus, knitr, rmarkdown, testthat License: Apache License (== 2.0) | file LICENSE NeedsCompilation: yes Package: assertthat Version: 0.2.0 Imports: tools Suggests: testthat License: GPL-3 NeedsCompilation: no Package: cli Version: 1.0.0 Depends: R (>= 2.10) Imports: assertthat, crayon, methods Suggests: covr, mockery, testthat, withr License: MIT + file LICENSE NeedsCompilation: no Package: crayon Version: 1.3.4 Imports: grDevices, methods, utils Suggests: mockery, rstudioapi, testthat, withr License: MIT + file LICENSE NeedsCompilation: no Package: pillar Version: 1.2.1 Imports: cli (>= 1.0.0), crayon (>= 1.3.4), methods, rlang (>= 0.2.0), utf8 (>= 1.1.3) Suggests: knitr (>= 1.19), lubridate, testthat (>= 2.0.0) License: GPL-3 NeedsCompilation: no Package: tidyselect Version: 0.2.4 Depends: R (>= 3.1) Imports: glue, purrr, rlang (>= 0.2.0), Rcpp (>= 0.12.0) LinkingTo: Rcpp (>= 0.12.0), Suggests: covr, dplyr, testthat License: GPL-3 NeedsCompilation: yes Package: tibble Version: 1.4.2 Depends: R (>= 3.1.0) Imports: cli, crayon, methods, pillar (>= 1.1.0), rlang, utils Suggests: covr, dplyr, import, knitr (>= 1.5.32), microbenchmark, mockr, nycflights13, rmarkdown, testthat, withr License: MIT + file LICENSE NeedsCompilation: yes Package: stringi Version: 1.1.7 Depends: R (>= 2.14) Imports: tools, utils, stats License: file LICENSE License_is_FOSS: yes NeedsCompilation: yes Package: rlang Version: 0.2.0 Depends: R (>= 3.1.0) Suggests: crayon, knitr, methods, pillar, rmarkdown (>= 0.2.65), testthat, covr License: GPL-3 NeedsCompilation: yes Package: Rcpp Version: 0.12.16 Depends: R (>= 3.0.0) Imports: methods, utils Suggests: RUnit, inline, rbenchmark, knitr, rmarkdown, pinp, pkgKitten (>= 0.1.2) License: GPL (>= 2) NeedsCompilation: yes Package: purrr Version: 0.2.4 Depends: R (>= 3.1) Imports: magrittr (>= 1.5), rlang (>= 0.1), tibble Suggests: covr, dplyr (>= 0.4.3), knitr, rmarkdown, testthat License: GPL-3 | file LICENSE NeedsCompilation: yes Package: magrittr Version: 1.5 Suggests: testthat, knitr License: MIT + file LICENSE NeedsCompilation: no Package: glue Version: 1.2.0 Depends: R (>= 3.1) Imports: methods Suggests: testthat, covr, magrittr, crayon, knitr, rmarkdown, DBI, RSQLite, R.utils, forcats, microbenchmark, rprintf, stringr, ggplot2 License: MIT + file LICENSE NeedsCompilation: yes Package: tidyr Version: 0.8.0 Depends: R (>= 3.2) Imports: dplyr (>= 0.7.0), glue, magrittr, purrr, Rcpp, rlang, stringi, tibble, tidyselect LinkingTo: Rcpp Suggests: covr, gapminder, knitr, rmarkdown, testthat License: MIT + file LICENSE NeedsCompilation: yes 5) Move the downloaded packages from #1 and #2 to the /tmp/ryantester/src/contrib 6) The final step is to install, pointing to your local repo (in our case, /tmp/ryantester) install.packages('tidyr', repos = "file:///tmp/ryantester") ... View more