这是我在 Amazon AMI 上运行 GUI 的解决方案。我用过这个post作为起点，但必须进行许多更改才能使其在 Amazon 的 AMI 上运行。我还添加了额外的信息，使这项工作以相当自动化的方式进行，因此需要多次启动此环境的个人可以轻松完成此操作。
注意：我在这篇文章中包含了很多评论。我提前道歉，但我认为，如果需要进行修改的人能够理解为什么一路上做出各种选择，这可能会对他们有所帮助。
下面包含的脚本会一路安装一些文件。有关这些脚本使用的文件和目录结构的列表，请参阅第 4 节。
步骤 1. 安装桌面
执行“yum update”后，大多数解决方案都包含类似这样的行
sudo yum groupinstall -y "Desktop"
这个看似简单的步骤需要在 Amazon AMI 上付出更多的努力。该组未在 Amazon AMI（从现在开始为 AAMI）中配置。默认情况下，AAMI 安装并启用了 Amazon 自己的存储库。还安装了 epel repo，但默认情况下它是禁用的。启用 epel 后，我找到了桌面组，但它没有填充软件包。我还发现了 Xfce（另一个桌面替代品），它已被填充。最终我决定安装 Xfce 而不是 Desktop。尽管如此，这并不简单，但最终找到了解决方案。
在这里值得注意的是，我尝试的第一件事是安装 centos 存储库并从那里安装桌面组。最初这似乎很有希望。小组里挤满了包裹。然而，经过一番努力，我最终发现 AAMI 上已安装的依赖项和软件包之间存在太多版本冲突。
这导致我从 epel 存储库中选择 Xfce。由于 epel 存储库已安装在 AAMI 上，我认为与 Amazon 存储库会有更好的依赖版本协调。这通常是正确的。在 epel 存储库或 Amazon 存储库中发现了许多依赖项。对于那些没有的，我可以在 centos 存储库中找到它们，并且在大多数情况下，这些都是叶依赖项。因此，大部分问题来自于 centos 存储库中的少数依赖项，这些依赖项具有与亚马逊或 epel 存储库冲突的子依赖项。最后需要一些技巧来绕过依赖冲突。我试图尽可能地减少这些。这是安装Xfce的脚本
安装Gui.sh
#!/bin/bash
# echo each command
set -x
# assumes RSRC_DIR and IS_EMR set by parent script
YUM_RSRC_DIR=$RSRC_DIR/yum
sudo yum -y update
# Most info I've found on installing a GUI on AWS suggests to install using
#> sudo yum groupinstall -y "Desktop"
# This group is not available by default on the Amazon Linux AMI. The group
# is listed if the epel repo is enabled, but it is empty. I tried installing
# the centos repo, which does have support for this group, but it simply end
# up having to many dependency version conflicts with packages already installed
# by the Amazon repos.
#
# I found the path of least resistance to be installing the group Xfce from
# the epel repo. The epel repo is already included in amazon image, just not enabled.
# So I'm guessing there was at least some consideration by Amazon to align
# the dependency versions of this repo with the Amazon repos.
#
# My general approach to this problem was to start with the last command:
#> sudo yum groupinstall -y Xfce
# which will generate a list of missing dependencies. The script below
# essentially works backwards through that list to eliminate all the
# missing dependencies.
#
# In general, many of the dependencies required by Xfce are found in either
# the epel repo or the Amazon repos. Most of the remaining dependencies can be
# found in the centos repo, and either don't have any further dependencies, or if they
# do those dependencies are satisfied with the centos repo with no collisions
# in the epel or amazon repo. Then there are a couple of oddball dependencies
# to clean up.
# if yum-config-manager is not found then install yum-utils
#> sudo yum install yum-utils
sudo yum-config-manager --enable epel
# install centos repo
# place the repo config @ /etc/yum.repos.d/centos.repo
sudo cp $YUM_RSRC_DIR/yum.repos.d/centos.repo /etc/yum.repos.d/
# The config centos.repo specifies the key with a URL. If for some reason the key
# must be in a local file, it can be found here: https://www.centos.org/keys/RPM-GPG-KEY-CentOS-6
# It can be installed to the right location in one step:
#> wget -O /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 https://www.centos.org/keys/RPM-GPG-KEY-CentOS-6
# Note, a key file must also be installed in the system key ring. The docs are a bit confusing
# on this, I found that I needed to run both gpg AND then followed by rpm, eg:
#> sudo gpg --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
#> sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
# I found there are a lot of version conflicts between the centos, Amazon and epel repos.
# So I did not enable the centos repo generally. Instead I used the --enablerepo switch
# enable it explicitly for each yum command that required it. This only works for yum. If
# rpm must be used, then yum-config-manager must be used to enable/disable repos as a
# separate step.
#
# Another problem I ran into was yum installing the 32-bit (*.i686) package rather than
# the 64-bit (*.x86_64) verision of the package. I never figured out why. So I had
# to specify the *.x86_64 package explicitly. The search tools (eg. 'whatprovides')
# did not list the 64 bit package either even though a manual search through the
# package showed the 64 bit components were present.
#
# Sometimes it is difficult to determine which package must be in installed to satisfy
# a particular dependency. 'whatprovides' is a very useful tool for this
#> yum --enablerepo centos whatprovides libgdk_pixbuf-2.0.so.0
#> rpm -q --whatprovides libgdk_pixbuf
sudo yum --enablerepo centos install -y gdk-pixbuf2.x86_64
sudo yum --enablerepo centos install -y gtk2.x86_64
sudo yum --enablerepo centos install -y libnotify.x86_64
sudo yum --enablerepo centos install -y gnome-icon-theme
sudo yum --enablerepo centos install -y redhat-menus
sudo yum --enablerepo centos install -y gstreamer-plugins-base.x86_64
# problem when we get to libvte, installing libvte requires expat, which conflicts with amazon lib
# the centos package version was older and did not install right lib version
# but … the expat dependency was coming from a dependency on python-libs.
# the easiest workaround was to install python using the amazon repo, that in turn
# installs a version of python libs that is compatible with the version of libexpat on the system.
sudo yum install -y python
sudo yum --enablerepo centos install -y vte.x86_64
sudo yum --enablerepo centos install -y libical.x86_64
sudo yum --enablerepo centos install -y gnome-keyring.x86_64
# another sticky point, xfdesktop requires desktop-backgrounds-basic, but ‘whatprovides’ does not
# provide any packages for this query (not sure why). It turns out this is provided by the centos
# repo, installing ‘desktop-backgrounds-basic’ will try to install the package redhat-logos, but
# unfortunately this is obsoleted by Amazon’s generic-logos package
# The only way I could find to get around this was to erase the generic logos package.
# This doesn't seem too risky since this is just images for the desktop and menus.
#
sudo yum erase -y generic-logos
# Amazon repo must be disabled to prevent interference with the install
# of redhat-logos
sudo yum --disablerepo amzn-main --enablerepo centos install -y redhat-logos
# next problem is a dependency on dbus. The dependency comes from dbus-x11 in
# centos repo. It requires dbus version 1.2.24, the amazon image already has
# version 1.6.12 installed. Since the dbus-x11 is only used by the GUI package,
# easiest way around this is to install dbus-x11 with no dependency checks.
# So it will use the newer version of dbus (should be OK). The main thing that could be a problem
# here is if it skips some other dependency. When doing manually, its possible to run the install until
# the only error left is the dbus dependency. It’s a bit risky running in a script since, basically it’s assuming
# all the dependencies are already in place.
yumdownloader --enablerepo centos dbus-x11.x86_64
sudo rpm -ivh --nodeps dbus-x11-1.2.24-8.el6_6.x86_64.rpm
rm dbus-x11-1.2.24-8.el6_6.x86_64.rpm
sudo yum install -y xfdesktop.x86_64
# We need the version of poppler-glib from centos repo, but it is found in several repos.
# Disable the other repos for this step.
# On EMR systems a newer version of poppler is already installed. So move up 1 level
# in dependency chain and force install of tumbler.
if [ $IS_EMR -eq 1 ]
then
yumdownloader --enablerepo centos tumbler.x86_64
sudo rpm -ivh --nodeps tumbler-0.1.21-1.el6.x86_64.rpm
else
sudo yum --disablerepo amzn-main --disablerepo amzn-updates --disablerepo epel --enablerepo centos install -y poppler-glib
fi
sudo yum install --enablerepo centos -y polkit-gnome.x86_64
sudo yum install --enablerepo centos -y control-center-filesystem.x86_64
sudo yum groupinstall -y Xfce
以下是 centos 存储库配置文件的内容：
centos.repo
[centos]
name=CentOS mirror
baseurl=http://repo1.ash.innoscale.net/centos/6/os/x86_64/
failovermethod=priority
enabled=0
gpgcheck=1
gpgkey=https://www.centos.org/keys/RPM-GPG-KEY-CentOS-6
如果您需要的只是在 Amazon AMI 上安装桌面软件包的方法，那么您就完成了。本文的其余部分将介绍如何配置 VNC 以通过 SSH 隧道访问桌面，以及如何打包所有这些内容，以便可以从脚本轻松启动实例。
步骤 2. 安装和配置 VNC
下面是我安装 GUI 的顶级脚本。配置几个变量后，它所做的第一件事就是调用上面步骤 1 中的脚本。该脚本有一些额外的负担，因为我将其构建为在常规 ec2 实例或 emr 上以 root 或 ec2 用户身份运行。基本步骤是
- 安装 libXfont
- 安装tigervnc服务器
- 安装VNC服务器配置文件
- 在用户主目录中创建.vnc目录
- 在.vnc目录中安装xstartup文件
- 在 .vnc 目录中安装虚拟 passwd 文件
- 启动VNC服务器
需要注意的几个要点：
这假设您将通过 SSH 隧道访问 VNC 服务器。最后，这确实看起来是最简单、最可靠、最安全的方法。由于您可能在安全组规范中打开了 SSH 端口，因此您无需对其进行任何更改。此外，VNC 客户端/服务器的加密配置并不简单。似乎很容易犯错误并使通信不加密。此设置位于 vncservers 文件中。 -localhost 开关告诉 vnc 仅接受本地连接。 “-nolisten tcp”告诉关联的 xserver 模块也不接受来自网络的连接。最后，“-SecurityTypes None”开关允许您在不输入密码的情况下打开 VNC 会话，因为进入计算机的唯一方法是通过 ssh，额外的密码检查似乎是多余的。
xstartup 文件确定首次启动 VNC 会话时将启动的内容。我注意到很多关于这个主题的帖子都跳过了这一点。如果你不告诉它启动 Xfce 桌面，那么当你启动 VNC 时，你只会看到一个空白窗口。我这里的配置非常简单。
尽管我上面提到 VNC 服务器被配置为不提示输入密码，但它仍然需要 .vnc 目录中的 passwd 文件才能启动服务器。第一次运行该脚本时，它尝试启动服务器时会失败。通过 ssh 登录计算机并运行“vncpasswd”。它将在 .vnc 目录中创建一个 passwd 文件，您可以保存该文件以在安装过程中用作这些脚本的一部分。请注意，我读到 VNC 没有做任何复杂的事情来保护 passwd 文件。因此，我不建议您使用用于其他更重要帐户的密码。
安装Gui.sh
#!/bin/bash
# echo each command
set -x
BIN_DIR="${BASH_SOURCE%/*}"
ROOT_DIR=$(dirname $BIN_DIR)
RSRC_DIR=$ROOT_DIR/rsrc
VNC_DIR=$RSRC_DIR/vnc
# Install user config files into ec2-user home directory
# if it is available. In practice, this should always
# be true
if [ -d "/home/ec2-user" ]
then
USER_ACCT=ec2-user
else
USER_ACCT=hadoop
fi
HOME_DIR="/home"
# Use existence of hadoop home directory as proxy to determine if
# this is an EMR system. Can be used later to differentiate
# steps on EC2 system vs EMR.
if [ -d "/home/hadoop" ]
then
IS_EMR=1
else
IS_EMR=0
fi
# execute Xfce desktop install
. "$BIN_DIR/installXfce.sh"
# now roughly follow the following from step 3: https://devopscube.com/setup-gui-for-amazon-ec2-linux/
sudo yum install -y pixman pixman-devel libXfont
sudo yum -y install tigervnc-server
# install the user account configuration file.
# This setup assumes the user will always connect to the VNC server
# through an SSH tunnel. This is generally more secure, easier to
# configure and easier to get correct than trying to allow direct
# connections via TCP.
# Therefore, config VNC server to only accept local connections, and
# no password required.
sudo cp $VNC_DIR/vncservers-$USER_ACCT /etc/sysconfig/vncservers
# install the user account, vnc config files
sudo mkdir $HOME_DIR/$USER_ACCT/.vnc
sudo chown $USER_ACCT:$USER_ACCT $HOME_DIR/$USER_ACCT/.vnc
# need xstartup file to tell vncserver to start the window manager
sudo cp $VNC_DIR/xstartup $HOME_DIR/$USER_ACCT/.vnc/
sudo chown $USER_ACCT:$USER_ACCT $HOME_DIR/$USER_ACCT/.vnc/xstartup
# Even though the VNC server is config'd to not require a passwd, the
# server still looks for the passwd file when it starts the session.
# It will fail if the passwd file is not found.
# The first time these scripts are run, the final step will fail.
# Then manually run
#> vncpasswd
# It will create the file ~/.vnc/passwd. Then save this file to persistent
# storage so that it can be installed to the user account during
# server initialization.
sudo cp $ROOT_DIR/home/user/.vnc/passwd $HOME_DIR/$USER_ACCT/.vnc/
sudo chown $USER_ACCT:$USER_ACCT $HOME_DIR/$USER_ACCT/.vnc/passwd
# This script will be running as root if called from the EC2 launch
# command. VNC server needs to be started as the user that
# you will connect to the server as (eg. ec2-user, hadoop, etc.)
sudo su -c "sudo service vncserver start" -s /bin/sh $USER_ACCT
# how to stop vncserver
# vncserver -kill :1
# On the remote client
# 1. start the ssh tunner
#> ssh -i ~/.ssh/<YOUR_KEY_FILE>.pem -L 5901:localhost:5901 -N ec2-user@<YOUR_SERVER_PUBLIC_IP>
# for debugging connection use -vvv switch
# 2. connect to the vnc server using client on the remote machine. When
# prompted for the IP address, use 'localhost:5901'
# This connects to port 5901 on your local machine, which is where the ssh
# tunnel is listening.
vnc服务器
# The VNCSERVERS variable is a list of display:user pairs.
#
# Uncomment the lines below to start a VNC server on display :2
# as my 'myusername' (adjust this to your own). You will also
# need to set a VNC password; run 'man vncpasswd' to see how
# to do that.
#
# DO NOT RUN THIS SERVICE if your local area network is
# untrusted! For a secure way of using VNC, see this URL:
# http://kbase.redhat.com/faq/docs/DOC-7028
# Use "-nolisten tcp" to prevent X connections to your VNC server via TCP.
# Use "-localhost" to prevent remote VNC clients connecting except when
# doing so through a secure tunnel. See the "-via" option in the
# `man vncviewer' manual page.
# Use "-SecurityTypes None" to allow session login without a password.
# This should only be used in combination with "-localhost"
# Note: VNC server still looks for the passwd file in ~/.vnc directory
# when the session starts regardless of whether the user is
# required to enter a passwd.
# VNCSERVERS="2:myusername"
# VNCSERVERARGS[2]="-geometry 800x600 -nolisten tcp -localhost"
VNCSERVERS="1:ec2-user"
VNCSERVERARGS[1]="-geometry 1280x1024 -nolisten tcp -localhost -SecurityTypes None"
xstartup
#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
# exec /etc/X11/xinit/xinitrc
/usr/share/vte/termcap/xterm &
/usr/bin/startxfce4 &
步骤 3. 连接到您的实例
在 EC2 上运行 VNC 服务器后，您可以尝试连接到它。首先打开到您的实例的 SSH 隧道。 5901 是 VNC 服务器侦听 vncservers 文件中的显示 1 的端口。它将侦听端口 5902 上的显示 2 等。此命令创建从本地计算机上的端口 5901 到实例上的端口 5901 的隧道。
ssh -i ~/.ssh/<YOUR_KEY_FILE>.pem -L 5901:localhost:5901 -N ec2-user@<YOUR_SERVER_PUBLIC_IP>
现在打开您首选的 VNC 客户端。在提示输入服务器 IP 地址的地方输入：
本地主机:5901
如果没有任何反应，则可能是启动 vnc 服务器时出现问题，或者存在阻止客户端访问服务器的连接问题，或者 vncservers 配置文件中可能存在问题
如果出现一个窗口，但它只是空白，则检查 Xfce 安装是否成功完成以及 xstartup 文件是否已安装。
步骤 4. 简化
如果您只需要执行一次此操作，那么将脚本通过 sftp 传输到您的实例并手动运行就可以了。否则，当您确实需要使用 GUI 启动实例时，您将希望尽可能地自动化此操作，以使其更快且不易出错。
自动化的第一步是创建一个 EFS 卷，其中包含可以在实例启动时安装的脚本和配置文件。亚马逊有很多info关于创建网络文件系统。创建卷时需要注意的几点。如果您不希望您的卷向全世界开放，您可能需要创建一个自定义安全组以用于您的 EFS 卷。我为 EFS 卷创建了安全组（称为 NFS_Mount），该安全组仅允许端口 2049 上来自我的其他安全组之一（称为 MasterVNC）的入站 TCP 流量。然后，当您创建实例时，请确保将 MasterVNC 安全组与该实例关联。否则 EFS 卷将不允许您的实例与其连接。
现在挂载 EFS 卷：
sudo mkdir /mnt/YOUR_MOUNT_POINT_DIR
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-YOUR_EFS_ID.efs.us-east-1.amazonaws.com:/ /mnt/YOUR_MOUNT_POINT_DIR
现在使用以下目录结构使用步骤 1 和 2 中提到的 6 个文件填充 /mnt/YOUR_MOUNT_POINT_DIR。请记住，您必须第一次使用命令“vncpasswd”创建 passwd 文件。它将在 ~/.vnc/passwd 中创建文件。
/mnt/YOUR_MOUNT_POINT_DIR/bin/installGui.sh
/mnt/YOUR_MOUNT_POINT_DIR/bin/installXfce.sh
/mnt/YOUR_MOUNT_POINT_DIR/rsrc/vnc/vncservers-ec2-user
/mnt/YOUR_MOUNT_POINT_DIR/rsrc/vnc/xstartup
/mnt/YOUR_MOUNT_POINT_DIR/rsrc/vnc/passwd
/mnt/YOUR_MOUNT_POINT_DIR/rsrc/yum/yum.repos.d/centos.repo
此时，使用 GUI 设置实例应该非常容易。像平常一样创建实例（确保包含 MasterVNC 安全组），通过 ssh 连接到实例，安装 EFS 卷，然后运行 installGui.sh 脚本。
步骤 5. 自动化
您可以更进一步，使用本地计算机上的 AWS CLI 工具一步启动实例。为此，您需要挂载 EFS 卷并使用 AWS CLI 命令的参数运行 installGui.sh 脚本。这只需要创建一个顶级脚本并将其传递给 CLI 命令。
当然，还有一些并发症。 EC2 和 EMR 使用不同的开关和机制来附加脚本。此外，在 EMR 上，我只希望 GUI 安装在主节点（而不是核心或任务节点）上。
启动 EC2 实例需要使用 --user-data 开关将脚本嵌入到命令中。通过指定本地计算机上脚本文件的绝对路径，可以轻松完成此操作。
aws ec2 run-instances --user-data file:///PATH_TO_YOUR_SCRIPT/top.sh ... other options
EMR 启动不支持从本地文件嵌入脚本。相反，您可以在引导操作中指定 S3 URI。
aws emr create-cluster --bootstrap-actions '[{"Path":"s3://YOUR_BUCKET/YOUR_DIR/top.sh","Name":"Custom action"}]' ... other options
最后，您将在下面的 top.sh 中看到大部分脚本是用于确定计算机是基本 EC2 实例还是 EMR 主实例的函数。如果不是这样，脚本可能只有 3 行。您可能想知道为什么不只使用内置的“run-if”引导操作而不是编写我自己的函数。内置的“run-if”脚本有一个错误，无法正确运行位于 S3 中的脚本。
一旦将它们放入初始化序列中，对其进行调试可能是一个挑战。可以提供帮助的一件事是日志文件：/var/log/cloud-init-output.log。这会捕获引导初始化期间运行的脚本的所有控制台输出。
top.sh
#!/bin/bash
# note: conditional bootstrap function run-if has a bug, workaround ...
# this function adapted from https://forums.aws.amazon.com/thread.jspa?threadID=222418
# Determine if we are running on the master node.
# 0 - running on master, or non EMR node
# 1 - running on a task or core node
check_if_master_or_non_emr() {
python - <<'__SCRIPT__'
import sys
import json
instance_file = "/mnt/var/lib/info/instance.json"
try:
with open(instance_file) as f:
props = json.load(f)
is_master_or_non_emr = props.get('isMaster', False)
except IOError as ex:
is_master_or_non_emr = True # file will not exist when testing on a non-emr machine
if is_master_or_non_emr:
sys.exit(1)
else:
sys.exit(0)
__SCRIPT__
}
check_if_master_or_non_emr
IS_MASTER_OR_NON_EMR=$?
# If this machine is part of EMR cluster, then ONLY install on the MASTER node
if [ $IS_MASTER_OR_NON_EMR -eq 1 ]
then
sudo mkdir /mnt/YOUR_MOUNT_POINT_DIR
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-YOUR_EFS_ID.efs.us-east-1.amazonaws.com:/ /mnt/YOUR_MOUNT_POINT_DIR
. /mnt/YOUR_MOUNT_POINT_DIR/bin/installGui.sh
fi
exit 0
