Apache Ranger 0.7.1 is the current version of Apache Ranger. This blog will help you to Install Apache Ranger 0.7.1 on CentOS operating system. I have explained the entire process in simple and easy steps.
Prerequisites
The following components are required to process ranger installation.
- JDK – For Running RangerAdmin, RangerKMS
- RDBMS – Storing policies, Ranger Users, and Groups (MySQL/Oracle/Postgres/SQL Server)
- Solr – Store Audit Logs (Apache Solr – 5.2.1+)
- HDFS – Store Audit Logs
- Kerberos – Enabling Kerberos will ensure that the requests get authenticated.
Preparing to install
You need to log in as root or will need “sudo” access to install some of the build tools.
Step 1: Install Maven 3.3.9
$ cd /usr/local # Download maven latest distribution tar from apache maven site $ wget http://www-eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz $ tar -xvf apache-maven-3.3.9-bin.tar.gz # Add environment varialves in .bashrc export M2_HOME=/usr/local/apache-maven-3.3.9 export M2=$M2_HOME/bin export PATH=$M2:$PATH #Now to test your install of Maven, $ mvn -version
Step 2: Install git
$ yum install git
Step 3: Install gcc
$ yum install gcc
Step 4: Install MySQL
- Download the release package for your platform from https://dev.mysql.com/downloads/repo/yum/
$ wget http://dev.mysql.com/get/mysql57-community-release-el6-7.noarch.rpm $ sudo rpm -Uvh mysql57-community-release-el6-7.noarch.rpm $ sudo yum install mysql-community-server $ service mysqld start
- To get the temp mysql password
$ grep 'temporary password' /var/log/mysqld.log
- Change default settings
mysql_secure_installation
- Download the MySQL JDBC and place it into shared director
$ wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.40.tar.gz $ tar -xvf mysql-connector-java-5.1.40.tar.gz $ mv mysql-connector-java-5.1.40/mysql-connector-java-5.1.40-bin.jar /usr/share/java/mysql-connector-java.jar
Build Ranger Admin from source
Add environment varialves in .bashrc
export MAVEN_OPTS="-Xmx2048m" export _JAVA_OPTIONS="-Xms1024m -Xmx1024m -XX:MaxPermSize=512m" export JAVA_HOME=/usr/lib/jvm/java-openjdk export PATH=$JAVA_HOME/bin:$PATH
Clone ranger source code
$ mkdir ~/dev $ cd ~/dev $ git clone https://github.com/apache/incubator-ranger.git $ cd incubator-ranger $ git checkout tags/release-0.7.1 -b ranger-0.7.1
Building the source
$ cd ~/dev/incubator-ranger
Run Maven to build ranger packages
$ mvn clean compile package assembly:assembly install
After maven build succeeds, Verify all tar files under target directory
$ ls target/*.tar.gz
Sample output:
ranger-0.7.1-admin.tar.gz ranger-0.7.1-kms.tar.gz ranger-0.7.1-storm-plugin.tar.gz ranger-0.7.1-hbase-plugin.tar.gz ranger-0.7.1-knox-plugin.tar.gz ranger-0.7.1-usersync.tar.gz ranger-0.7.1-hdfs-plugin.tar.gz ranger-0.7.1-migration-util.tar.gz ranger-0.7.1-yarn-plugin.tar.gz ranger-0.7.1-hive-plugin.tar.gz ranger-0.7.1-solr-plugin.tar.gz ranger-0.7.1-kafka-plugin.tar.gz ranger-0.7.1-src.tar.gz
Install Solr
Audit logs are stored in Solr. (Note: if don’t want Solr, we can use DB to store the log https://community.hortonworks.com/questions/27957/does-ranger-05-work-without-solr.html) (we will try for store logs in HDFS)
$ cd ~/dev/incubator-ranger/security-admin/contrib/solr_for_audit_setup
Edit install.properties and add the following values,
SOLR_INSTALL = true SOLR_DOWNLOAD_URL = http://archive.apache.org/dist/lucene/solr/5.2.1/solr-5.2.1.tgz SOLR_INSTALL_FOLDER = /opt/solr JAVA_HOME = /usr/lib/jvm/java-openjdk SOLR_USER = solr SOLR_RANGER_HOME = /opt/solr/ranger_audit_server SOLR_RANGER_PORT = 6083 SOLR_DEPLOYMENT = standalone SOLR_RANGER_DATA_FOLDER = /opt/solr/ranger_audit_server/data SOLR_LOG_FOLDER = /var/log/solr/ranger_audits SOLR_MAX_MEM = 2g
Run setup script to install solr
$ ./setup.sh
Open /opt/solr/ranger_audit_server/install_notes.txt for instructions to start and stop Solr.
Check Solr by accessing http://${SOLR_HOST}:6083 from your browser.
Install Ranger Policy Admin
$ cd /usr/local $ sudo tar zxvf ~/dev/incubator-ranger/target/ranger-0.7.1-admin.tar.gz $ sudo ln -s ranger-0.7.1-admin/ ranger-admin $ cd /usr/local/ranger-admin/
Configure Solr
- Update install.properties
$ cd /usr/local/ranger-admin $ vi install.properties # Mysql root db_root_user=root db_root_password=root db_host=localhost # DB UserId used for the XASecure schema db_name=ranger db_user=rangeradmin db_password=rangeradmin # audit log audit_store=solr audit_solr_urls=http://HOST_ADDRESS:6083/solr/ranger_audits policymgr_external_url=http://localhost:6080
- After updating the required properties, run setup.sh
$ ./setup.sh
- Now th following command starts Ranger Admin
$ ranger-admin start
The ranger-admin script supports the following options,
- Starting Ranger Admin service: ranger-admin start
- Stopping Ranger Admin service: ranger-admin stop
- Restarting Ranger Admin service: ranger-admin restart
Verification can be done visiting the external URL of the server using browser, for instance
http://HOST_ADDRESS:6080 (Username: admin & PAssword: admin)
Configuring Ranger Admin Authentication Modes
- Active Directory, LDAP and Unix are the authentication methods supported by Ranger Admin.
UNIX
authentication_method = UNIX
remoteLoginEnabled = true
authServiceHostName = localhost
authentication_method = 5151
Installing the Ranger UserSync Process
- Extract Ranger plugin build at /usr/local
$ cd /usr/local $ tar zxvf ~/dev/incubator-ranger/target/ranger-0.7.1-usersync.tar.gz $ ln -s ranger-0.7.1-usersync/ ranger-usersync $ cd ranger-usersync $ mkdir -p /var/log/ranger-usersync $ chown ranger /var/log/ranger-usersync $ chgrp ranger /var/log/ranger-usersync
- Edit the install.properties file
$ POLICY_MGR_URL = http://HOST_ADDRESS:6080 $ SYNC_SOURCE = unix $ logdir = /var/log/ranger/usersync
- Install usersync by running ./setup.sh command
$ cd /usr/local/ranger-usersync $ ./setup.sh
- After installing ranger usersync, Use start/stop services of usersync.
./ranger-usersync-services.sh start ./ranger-usersync-services.sh stop
Apache Ranger HDFS Plugins
Step 1: Create new REPOSITORY_NAME service
If the REPOSITORY_NAME service doesn’t exist in Ranger Admin, the hdfs-plugin cannot communicate with it. So we have to make sure that it exists.
- Go to http://HOST_ADDRESS:6080
- Open Resource based policies link
- Click ‘+’ under HDFS Grid to add new service
- Update Service and Config Properties with the following
Service Name : hadoopdev
Active Status : Enabled
Select Tag Service : HDFS (Service name created under Tag based policies)
Username : hadoop
Password : hadoop@123
Namenode URL : http://HOST_ADDRESS:50070
- Leave all other field as defaults and save
Step 2: Install and configure HDFS Plugins
- Extract Ranger plugin build at /usr/local
$ cd /usr/local $ tar zxvf ~/dev/incubator-ranger/target/ranger-0.7.1-hdfs-plugin.tar.gz $ ln -s ranger-0.7.1-hdfs-plugin/ ranger-hdfs-plugin $ cd ranger-hdfs-plugin
- Edit the install.properties file
POLICY_MGR_URL = http://HOST_ADDRESS:6080 REPOSITORY_NAME = hadoopdev XAAUDIT.SOLR.ENABLE = true XAAUDIT.SOLR.URL = http://HOST_ADDRESS:6083/solr/ranger_audits XAAUDIT.SOLR.USER = NONE XAAUDIT.SOLR.PASSWORD = NONE XAAUDIT.SOLR.ZOOKEEPER = NONE XAAUDIT.SOLR.FILE_SPOOL_DIR = /var/log/hadoop/hdfs/audit/solr/spool
- Enable hdfs-plugin by running ./enable-hdfs-plugin.sh command
$ cd /usr/local/ranger-hdfs-plugin $ ./enable-hdfs-plugin.sh
Ranger hdfs plugin installation might fail. To resolve this issue create a symlink as conf dir of hadoop linking to hadoop conf dir.
$ cd /usr/local/hadoop/ $ ln -s /home/hadoop/hadoop/etc/hadoop/ /usr/local/hadoop/conf $ ln -s /home/hadoop/hadoop/share/hadoop/hdfs/lib/ /usr/local/hadoop/lib
- Add HADOOP_HOME env value into .bashrc file
export HADOOP_HOME=/home/hadoop/hadoop
- Once the above changes are done run enable plugin script
$ cd /usr/local/ranger-hdfs-plugin $ ./enable-hdfs-plugin.sh
Ranger Hive Plugin
Step 1: Create new REPOSITORY_NAME service
Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not, the hive-plugin will not be able to communicate with Ranger admin
- Go to http://HOST_ADDRESS:6080
- Open Resource based policies link
- Click + under HIVE Grid to add new service
- Update following Service Details and Config Properties
Service Name : hivedev
Active Status : Enabled
Select Tag Service : HIVE (Service name created under Tag based policies)
Username : hadoop
Password : hadoop@123
jdbc.driverClassName : org.apache.hive.jdbc.HiveDriver
jdbc.url : jdbc:hive2://sandbox:10000/
- Leave all other field as default and save
Step 2: Install and configure HIVE Plugins
- Extract Ranger plugin build at /usr/local
$ cd /usr/local $ tar zxvf ~/dev/incubator-ranger/target/ranger-0.7.1-hive-plugin.tar.gz $ ln -s ranger-0.7.1-hive-plugin/ ranger-hive-plugin $ cd ranger-hive-plugin
- Edit the install.properties file
POLICY_MGR_URL = http://HOST_ADDRESS:6080 REPOSITORY_NAME = hivedev XAAUDIT.SOLR.ENABLE = true XAAUDIT.SOLR.URL = http://HOST_ADDRESS:6083/solr/ranger_audits XAAUDIT.SOLR.USER = NONE XAAUDIT.SOLR.PASSWORD = NONE XAAUDIT.SOLR.ZOOKEEPER = NONE XAAUDIT.SOLR.FILE_SPOOL_DIR = /var/log/hive/audit/solr/spool
- Create log directory
$ mkdir /var/log/hive $ chown -R hadoop:hadoop /var/log/hive
- Enable hive-plugin by running enable-hive-plugin.sh command
$ cd /usr/local/ranger-hive-plugin $ ./enable-hive-plugin.sh
That’s all now the latest version of Apache Ranger is installed in your system. Please comment below if you have any queries regarding the installation process. To read more on web development follow Agiratechnologies.
