Hue

big_data

发布日期: 2020-04-17

更新日期: 2020-04-26

字数: 1.2k

阅读时长: 6 分钟

阅读量:

[toc]

概述

安装运行

下载安装

CDH版本： http://archive.cloudera.com/cdh5/
官网：http://gethue.com/
tar -zxf hue-3.9.0-cdh5.15.0.tar.gz -C /opt/modules/

编译

下载解压之后要进行编译之后才能用，且当前网络环境要能访问外网。

安装各种依赖包

$ sudo yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c++ krb5-devel libffi-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite-devel gmp-devel

mvn (from apache-maven package or maven3 tarball)
libtidy (for unit tests only)
openssl-devel (for version 7+)

进入hue安装目录输入命令make apps

make[1]: Leaving directory `/opt/modules/hue-3.9.0-cdh5.15.0/apps'

基本配置

修改配置文件

#desktop/conf/hue.ini
#密钥-官网copy
secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn < qW5o
#host port
http_host=bigdata-pro03.bigDAta.com
http_port=8888
#时区
time_zone=Asia/Shanghai

修改desktop.db文件权限
```
chmod o+w desktop/desktop.db
```

服务启动

运行/opt/modules/hue-3.9.0-cdh5.15.0/build/env/bin/supervisor

切换回原来的网络，否则会报错cannot assign requested address

starting server with options:
{'daemonize': False,
 'host': 'bigdata-pro03.bigDAta.com',
 'pidfile': None,
 'port': 8888,
 'server_group': 'hue',
 'server_name': 'localhost',
 'server_user': 'hue',
 'ssl_certificate': None,
 'ssl_certificate_chain': None,
 'ssl_cipher_list': 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA',
 'ssl_private_key': None,
 'threads': 50,
 'workdir': None}

出现上述log，在浏览器输入bigdata-pro03.bigDAta.com:8888

第一次进入，会要求创建一个账户。

集成

官网链接

http://archive.cloudera.com/cdh5/cdh/5/hue-3.9.0-cdh5.5.0/manual.html

与HDFS集成

修改hadoop中core-site.xml配置文件(集群中所有机器都需要配置)

#core-site.xml
<property>
    <name>hadoop.proxyuser.hue.hosts</name>
    <value>*</value>
</property>
<property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>*</value>
</property>

修改hue.ini配置文件

#在hue.ini文件中找到[hadoop]->[[hdfs_clusters]]->[[[default]]]
fs_defaultfs=hdfs://ns
webhdfs_url=http://bigdata-pro01.bigDAta.com:50070/webhdfs/v1
hadoop_hdfs_home=/opt/modules/hadoop-2.6.0
hadoop_bin=/opt/modules/hadoop-2.6.0/bin
hadoop_conf_dir=/opt/modules/hadoop-2.6.0/etc/hadoop

启动hue

先启动zookeeper，hdfs，再启动hue

与YARN集成

修改hue.ini配置文件(需要区分集群是否是HA)

#在hue.ini文件中找到[hadoop]->[[yarn_clusters]]
#HA集群
[[[default]]]
      # Enter the host on which you are running the ResourceManager
      resourcemanager_host=rs
      # The port where the ResourceManager IPC listens on
      resourcemanager_port=8032
      submit_to=True
      #Resource Manager logical name (required for HA)
      logical_name=rm1
      resourcemanager_api_url=http://bigdata-pro01.bigDAta.com:8088
      proxy_api_url=http://bigdata-pro01.bigDAta.com:8088
      history_server_api_url=http://bigdata-pro01.bigDAta.com:19888
[[[ha]]]
      logical_name=rm2
      submit_to=True
      resourcemanager_api_url=http://bigdata-pro02.bigDAta.com:8088
      history_server_api_url=http://bigdata-pro01.bigDAta.com:19888
------------------------------------------------------------------------------
#非HA集群
[[[default]]]
      resourcemanager_host=rs
      resourcemanager_port=8032
      submit_to=True
      resourcemanager_api_url=http://bigdata-pro01.bigDAta.com:8088
      proxy_api_url=http://bigdata-pro01.bigDAta.com:8088
      history_server_api_url=http://bigdata-pro01.bigDAta.com:19888

<!--resourcemanager_host对应于yarn-site.xml中配置的-->
<property>
    <name>yarn.resourcemanager.cluster-id</name>
    <value>rs</value>
</property>
<!--logical_name名字对应于yarn-site.xml中配置的--> 
<property> 
    <name>yarn.resourcemanager.ha.rm-ids</name>  
    <value>rm1,rm2</value> 
</property> 
<!--resourcemanager_api_url配置资源管理的地址和端口，对应于yarn-site.xml中配置的-->
<property>
    <name>yarn.resourcemanager.webapp.address</name>
    <value>bigdata-pro01.bigDAta.com:8088</value>
</property>
<!--history_server_api_url配置历史记录资源管理的地址和端口，对应mapred-site.xml-->
<property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>bigdata-pro01.bigDAta.com:19888</value>
</property>

与Hive集成

修改hue.ini配置文件

[beeswax]
      # Host where HiveServer2 is running.
      hive_server_host=bigdata-pro03.bigDAta.com
      # Port where HiveServer2 Thrift server runs on.
      hive_server_port=10000
      # Hive configuration directory, where hive-site.xml is located
      hive_conf_dir=/opt/modules/hive-2.1.0/conf

与MySql集成

修改hue.ini配置文件

[[[mysql]]]
      nice_name="My SQL DB"
      # For MySQL and PostgreSQL, name is the name of the database.
      # For Oracle, Name is instance of the Oracle server. For express edition
      name=metastore
      engine=mysql
      host=bigdata-pro01.bigDAta.com
      port=3306
      user=root
      password=123456

<!--出现了database is locked异常-->
1.Hue的元数据库是sqlite，容易产生database is locked的异常,那就是mysql没有配置成功，查看上述配置，果然engine前的##忘记去掉了。
<!--"Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)")-->
2.不知怎么回事，host前面的##也没去掉
mysql有两种连接方式
  a.通过socket连接，该方式支持本机连接，需要在my.cnf文件中配置正确的socket，配置项：socket 配置    内容：/tmp/mysql.sock
    执行方式:mysql -u [用户名] -p [密码]
    b.通过tcp连接，该方式支持远程连接
    执行方式:mysql -h [数据库地址] -u [用户名] -p [密码]

与Hbase集成

修改hue.ini配置文件

[hbase]
   hbase_clusters=(Cluster|bigdata-pro01.bigDAta.com:9090)
   # HBase configuration directory, where hbase-site.xml is located.
   hbase_conf_dir=/opt/modules/hbase-1.0.0-cdh5.4.0/conf
   thrift_transport=buffered