RPM - DNF

DNF (Dandified YUM)

DNF is the next version of  YUM (YUM is a package manager for RPMs). 

dnf roughly maintains CLI compatibility with yum and existing AIX Toolbox repositories created for yum are working good with dnf too, so no changes needed in repository side.

yum is based on python2 and python2 is out of support, so there was a need to move to python3, dnf  works with python3.

AIX Toolbox News: https://www.ibm.com/support/pages/node/6833478

DNF install: https://community.ibm.com/community/user/power/blogs/sangamesh-mallayya1/2021/05/28/dnf-is-now-available-on-aix-toolbox

DNF config details: https://developer.ibm.com/tutorials/awb-configuring-dnf-create-local-repos-ibm-aix/

Just in case if all rpm need to be removed: https://community.ibm.com/community/user/power/blogs/jan-harris1/2022/05/25/destroyrpms

On AIX dnf uses the repository conf file: /opt/freeware/etc/dnf/dnf.conf

[AIX_Toolbox]

name=AIX generic repository

baseurl= file:///export/rpms/AIX_Toolbox/

enabled=1

...

...

[AIX_Toolbox_73]

name=AIX 7.3 specific repository

baseurl= file:///export/rpms/AIX_Toolbox_73/

enabled=0

[CUST_RPMS]

name=Customer specific RPMs

baseurl= file:///export/rpms/CUST_RPMS

enabled=1

By default dnf will cache data to the /var/cache/dnf directory, such as package and repository data. This speeds up dnf so that it doesn’t have to keep querying this information from the Internet.

There are times when you may want to delete this cached data, such as if a repository has updated packages but your system has incorrect or stale cached data which may cause various problems when attempting to install a package: dnf clean all

The dnf cache will be automatically built up over time when you perform various dnf queries such as installing or updating packages, however we have the option to manually make the cache so that future actions will be quicker with the ‘makecache’ argument: dnf makecache

-------------------------------------

DNF options

Special options can be used with the dnf commands, for example: dnf install httpd --verbose

some dnf options:

--cacheonly      <--run from system cache, don’t update the cache and use it even it is expired. (DNF uses a separate cache for each user, the root user cache is the system cache. )

--disablerepo / enablerepo=<repoid>   <--temporarily disable/enable  active repositories for the purpose of the current dnf command.

--downloadonly                   <--download without performing any rpm transaction (install/upgrade/erase).

--downloaddir / destdir=<path>   <-- download packages to this dir (it has to be used with --downloadonly, the download, modulesync, reposync or system-upgrade commands (dnf-plugins-core).

--exclude=<package-file-spec>    <--exclude packages specified by <package-file-spec> from the operation.

--nogpgcheck                     <--skip checking GPG signatures on packages (if RPM policy allows).

--refresh                        <--set metadata (cache) as expired before running the command.

--repo / repoid=<repoid>         <--enable just specific repositories by an id or a glob.

--showduplicates                 <--show duplicate packages in repositories. Applicable for the list and search commands.

--verbose                        <--Verbose operation, show debug messages.

--version                        <--Show DNF version and exit.  (-v can be used aswell)

--assumeyes                      <--Automatically answer yes for all questions. (-y can be used as well)

--setopt=<parameter=value>       <--override the configuration file (for example: --setopt=sslverify=false)

-------------------------------------

DNF Plugins

The core DNF functionality can be extended with plugins. 

There are officially supported Core DNF plugins and also third-party Extras DNF Plugins.

dnf-plugins-core  can be installed on Linux to have Core DNF plugins: download, repomange, reposync...

dnf download ...       <--download binary (rpm) or source packages without installing it (!!! this is missing on AIX)

dnf repomanage ...     <--repomanage prints newest or older packages in a repository specified by <path> for easy piping to xargs or similar programs.

dnf repodiff ...       <--list of differences between two or more repositories

dnf reposync ...       <--makes local copies of remote repos (sync a remote repo to a local dir, packages that are already present locally are not downloaded again)

On AIX most of plugins are in dnf-utils. dnf-plugins-core on AIX is not a real rpm, it creates dependencies for example to this: python3.9-dnf-plugins-core-4.0.16-32_52.aix7.2.ppc.rpm, which contains python files of repodiff, reposync...

# dnf repoquery -l dnf-utils

...

/opt/freeware/bin/repodiff

/opt/freeware/bin/repomanage

/opt/freeware/bin/repoquery

/opt/freeware/bin/reposync

/opt/freeware/libexec/dnf-utils

!!! createrepo command comes from the separate createrepo_c package not as dnf plugin (it is not in dnf-utils)

======================================================

/opt/freeware/etc/dnf/dnf.conf                    <--main dnf configuration file (dnf commands by default use this file)

dnf repolist                                      <--list repositories

dnf search bash*                                  <--list packgages starting with bash... in the repo

dnf list bash*                                    <--list packgages starting with bash... which are installed + in the repo

dnf list installed                                <--list installed packages (a package is installed if it is in RPMDB, same as rpm -qa)

dnf list installed x*                             <--list installed packages starting with "x"

dnf list available                                <--list pacckages that are available to install (a package is available if it is not installed but present in a repo)

dnf list upgrades                                 <--list updates available for installed packages ("update", "updates" are depreciated)

dnf check-upgrade                                 <--same as above

Officially "installed", "available"... actions should have 2 dash (--) in front, like --installed, --available...

A dnf list command should look like: dnf [options] list --installed (e.g.: dnf --config=/tmp/dnf.conf.remote list --upgrades)

By default "dnf list" uses the "--all" option, which list all packages present in rpmdb, in a repo or both (installed + available = full repo content)

dnf install <package>                             <--install a package + dependencies (more packages: dnf install package1 package2 …)

dnf install <package> -y                          <--install a package without asking anything before install (assumes yes)

dnf install <package> -v                          <--install with verbose output

dnf install <package_name>-<version_info>         <--install a specific version (like: dnf install gcc-6.3.0-1)

dnf localinstall </path/to/package>               <--install a package from local path instead of a repository

dnf install httpd-1.4.rpm                         <--install a local rpm file with dnf

dnf reinstall httpd                               <--if a package has a problem it can be reinstalled

dnf remove <package>                              <--remove a package

dnf upgrade                                       <--upgrade all possible installed packages ("update" is depreciated)

dnf upgrade <package>                             <--upgrde a package (with its dependencies if needed) 

dnf upgrade -x httpd                              <--exclude httpd packeage from the update

dnf downgrade <package>                           <--downgrade a package

dnf history                                       <--lists history of yum actions (same as yum history list all)

dnf history info <transaction_ID>                 <--gives details about the specified history transaction id

dnf history undo <transaction_ID>                 <--roll backs the given tranaction id

dnf info bash                                     <--show infow about specific package

dnf provides /opt/freeware/bin/bash               <--list the package which provides that file (command)  ("dnf repoquery --file..." or "rpm -qf ..." show some info as well)

dnf repoquery -l bash                             <--list files in a package (--list or "rpm -ql bash" is the same if it is already installed)

dnf makecache                                     <--download and caches metadata for repositories

dnf makecache --refresh

dnf clean all                                     <--cleans up cache

dnf --config /tmp/dnf.conf check-upgrade          <--list for available updates for our installed packages  (it will not do the update)

dnf --disablerepo="*" --enablerepo="epel" list available       <--lists packages only in a specific repo (use output of "yum repolist" )

dnf --disablerepo=* --enablerepo=LIVE* list Centrify*          <--lists installed and available packages from LIVE* repos

Plugins:

createrepo --checksum sha --update /etc/repo      <--update repo after a new package is copied there (dnf createrepo ... should work as well)

createrepo --quiet --update --skip-stat /export/rpms/AIX_Toolbox_72

--quiet                                           <--run quietly

--update                                          <--if metadata exists and rpm is unchanged (based on file size and mtime) then reuse the existing metadata rather then recalculating it

--skip-stat                                       <--skip stat() function call on files using --update (assumes if the file name is the same the the file is still the same)

dnf download httpd                                <--!!NOT on AIX!!! download the rpm without installing it (download plugin can be installed: dnf install dnf-plugins-core)

dnf repomanage --old /export/rpms/AIX_Toolbox     <--list older packages

dnf repomanage --new /export/rpms/AIX_Toolbox     <--list newest packages

download (sync) a repo locally from a conf file (this conf file contains the ibm repo address):

dnf reposync --newest-only --downloadcomps --download-metadata --download-path=/export/rpms --config=/tmp/dnf.ibm.conf --repoid=AIX_Toolbox_72 --arch=ppc

same as above just --urls will not download anything it will just show the urls for the packages:

dnf reposync --newest-only --downloadcomps --download-metadata --download-path=/home/tmp/dnf --config=/tmp/dnf.ibm.conf --repoid=AIX_Toolbox --arch=ppc --urls

dnf reposync \                    

--newest-only \                   <--download only newest packages per repo

--downloadcomps \                 <--download and uncompress comps.xml. Consider using --download-metadata which downloads all available repo metadata

--download-metadata \             <--download repository metadata. Downloaded copy is instantly usable as a repository, no need to run createrepo_c on it

--download-path=/export/rpms/ \   <--path under which the downloaded repositories are stored

--config=/opt/freeware/etc/dnf/dnf.conf.remote \              <--config file to use

--repoid=AIX_Toolbox_72           <--which repo to synchronize

--arch=ppc                        <--download packages of given architectures

repodiff

dnf repodiff --repofrompath=o,file:///export/rpms/AIX_Toolbox/ --repofrompath=n,https://public.dhe.ibm.com/aix/aixtoolbox/RPMS/ppc/ --repo-old=o --repo-new=n

repodiff without ssl verification:

dnf repodiff --setopt=sslverify=false --repofrompath=o,file:///export/rpms/AIX_Toolbox/ --repofrompath=n,https://public.dhe.ibm.com/aix/aixtoolbox/RPMS/ppc/ --repo-old=o --repo-new=n

DEVOPS - GIT

Git Basics

The purpose of Git is to keep track of the changes, which have been made on files. In more official wording: Git is a distributed version control system. Version Control System can be any software that records changes to files over time so that you can recall specific versions later. It helps software teams to keep track of modifications to the source code. If a mistake is made, developers can turn back the clock and compare earlier versions of the code to help fix the mistake. (Git was created by Linus Torvalds in 2005 to help during the development of the Linux Kernel with other kernel developers.)

Git is distributed, which means everyone has a local copy of all the files, so they can work independently of a central server. (In opposite, a Central Version Control system is  located in one place, and people can check out from the central location, make changes and check everything back in.)

When someone completes a task, he needs to do a commit. Commit is a snapshot of the state of your files, plus some metadata (such as who made the commit and when, with a comment, and a pointer to the previous commits). Every time you commit, or save the state of your project, Git basically takes a picture of what all your files look like at that moment and stores a reference to that snapshot. As time goes by, each commit is pointing to its parent to keep track all the changes that have been made:

Arrows are identifying a commit's parent (pointing to the left)

Some terminology:
-repository - this is where git stores all details, metadata which is needed for tracking (.git directory in the project dir)
-commit     - record changes to the repository (the local one)
-branch     - like in a tree, another path is taken in the main line of development, so the main line keeps untouched
-merge      - will take the independent lines of development (branches) and integrate them into a single branch
-master     - the repository’s main branch, usually other branches are checked out from this
-clone      - copies an existing git repository, which we don't have, (it is usually used only once)
-fetch      - downloads commits, files ...from a remote repository into your local repo and updates that local copy
-pull       - fetch (download) content from a remote repository and update local repository. (git fetch followed by git merge)
-push       - is used to submit the code to a remote repository
-head       - is a reference which points to the last commit we have made.
-checkout   - switch to another branch. To prepare working on a branch. (Pointing the head to that branch)

----------------------------------------------

Repository (.git directory)

Git stores all necessary information, metadata (which is needed to track all changes) in a data structure called a repository. The Git repository is in the same directory as the project itself, in a subdirectory called .git. You can obtain a Git repository in one of two ways:

1. You can take a local directory that is currently not under version control, and turn it into a Git repository.
You can go to any directory and use the command: git init. This creates a new subdirectory named .git that contains all of your necessary repository files. At this point, nothing in your project is tracked yet. If you want to start version-controlling existing files , you need to track those files (with git add) and do an initial commit (git commit).

2. You can clone an existing Git repository from elsewhere.
If you want to get a copy of an existing Git repository, "git clone" needs to be used. Then Git receives a full copy of all data that the server has. Every version of every file for the history of the project is pulled down by default

git clone <url>                         <--clone a repository to the current directory
git clone <url> <new name>              <--clone the repository into a directory named something other

----------------------------------------------

Working with Git

During our work in Git (using specific Git commands) our files can be in different states:
-Committed: It means that the data is safely stored in your local database.
-Staged: It means that you have marked a modified file in its current version to go into your next commit.
-Modified: It means that you have changed the file but have not committed it to your database yet.

The basic Git workflow goes something like this: 
You switch to the project you would like to work with command: git checkout. The you start to work in your working directory and modify files there. After finishing some work, you mark the files you have been worked on with command: git add. This will stage those files to be part of the next commit. In other words it will add those changes (files) to the staging area. After that when you do a commit, it will take the files as they are in the staging area and stores that snapshot permanently to your Git directory.

The .git directory is where Git stores the metadata and object database for your project. This is the most important part of Git, and it is what is copied when you clone a repository from another computer. If a particular version of a file is in the Git directory, it’s considered committed.

The staging area is a file, generally contained in your .git directory, that stores information about what will go into your next commit. If a file has been modified and was added to the staging area (with the command git add), it is staged.

The working directory (sometimes it is called working tree) is a single checkout of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify.

Everything in Git is check-summed before it is stored (committed) and is then referred to by that checksum. This means it’s impossible to change the content of a file or directory without Git knowing about it. The mechanism that Git uses for this checksumming is called a SHA-1 hash. This is a 40-character string.

 ----------------------------------------------

Starting to work with git

After install (like yum install git) some configuration is needed, for example to add your name to the checkouts and changes.
Config details can be seen by "git config --list" and changing configs can be done:

git config --global user.name "<user name>"
git config --global user.email "<user email>"

After that we can start to initialize a directory to be tracked by git:
1. go to the directory you would like to track
2. git init (it will create a .git directory which contains all details needed for tracking)
3. git status (show status of files if those are up to date, tracked etc..)

If there are files we don't want to track create .gitignore file:
# vi .gitignore:
.filename
*.filename

If this file exists "git status" will not show that file.

----------------------------------------------

Adding files to Staging area

When a new file is created it is not tracked automatically by Git. Git needs to be informed directly to track that file with git add. If you modify the file after you run git add, you have to run git add again to stage the latest version of the file. Otherwise the version when it was when you last ran the git add command will go into the commit, not the version of the file as it looks now.

This message can pop up in this case: “Changes not staged for commit”:
It means that a file that is tracked has been modified in the working directory but not yet staged. To stage it, you need to the git add command, which is a multipurpose command. You use it to begin tracking new files, to stage files, and to do other things. It may be helpful to think of it more as "add precisely this content to the next commit"

----------------------------------------------

Committing files:

When your staging area is set up, you can commit your changes. The usage is, git commit -m "Some message". If -m is omitted an editor will be popped up to put commit message there. With the commit, which will do a snapshot of the actual state of the staged files, these things will be saved: who did the commit, commit date, what SHA-1 checksum the commit has, what is the commit message etc.
(The commit records the snapshot you set up in your staging area. Anything you didn’t stage is still sitting there modified.)

1. git add -A                   <--add everything to the staging area
2. git status                   <--show the status
3. git diff                     <--shows changes made to the code
3. git commit -m "message"      <--committing files in the staging area (adds file to the repository)
4. git log                      <-- will show the commit what we made (hash number, name , date, message)

(git commit command commits only to the local branch we are working on and has no effect on other local branches, and it has no effect on remote repository as well)

----------------------------------------------

Commands:

git config --help                <--help
git --version                    <--check version
git init                         <--turn a directory into git repository (.git subdir is added with metadata)
rm -rf .git                      <--stop git tracking that directory by git (remove .git subdirectory)
.gitignore                       <--this file contains all the files which will be ignored from git
                                 
git status                       <--show the actual status of the working dir
git log                          <--shows commit history with hashes, dates, messages
git log --stat                   <--shows which files have been changed and part of the commit
git reflog                       <--shows in order check-outs, commits, resets

git add <file>                   <--adds a files to tracking (or if it is tracked puts in staging, which is needed before commit)
git add -A                       <--adds all files in current dir for tracking or to the staging area (git add . is the same)
git mv <file_old> <file_new>     <--rename a file in Git

git rm --cached  <file>          <--remove a file from git tracking (keeps the file on your hard drive but Git will not track)
git rm <file>                    <--remove a file from git tracking and from working dir (deletes from hard drive)(commit is needed)
git rm -f <file>                 <--if file is already added to staging area, then  -f option (force) is needed for removal
                                 
git reset <filename>             <--remove files from the staging area, unstaging a staged file
git reset                        <--will remove everything from the staging area

git commit -m "<message>"            <--commit  with a message
git commit --amend -m "new message"  <--change commit message to a new message
git commit --amend                   <--if we missed a file from last commit and want that to be part of that last commit
                                     (it brings up a vi editor, just save it: wq)

git branch                       <--list all local branches (* will show which branch we are currently working on)
git branch --merged              <--lists the branches that have been merged already
git branch -a                    <--lists all the branches of the repository (not only locally, remotely as well)
git branch -r                    <--lists all remote branches
git branch <new branch name>     <--create a branch
git branch -d <branch>           <--deletes branch locally

git checkout <branch name>       <--changing to the given branch (so we that given branch
git checkout --<filename>        <--if a file is modified and we want to dismiss modifications and go back to the original state

----------------------------------------------

git remote                       <--lists remote servers you have configured (shortname)
git remote -v                    <--shows you the URLs that Git has stored for the shortname
git remote show <remote>         <--show more information about a particular remote

git remote add <shortname> <url>  <--add a new remote Git repository as a shortname to reference
git remote add origin <url>       <--will create new remote (called origin) located at the specified link
git remote rename <current> <new> <--change a remote’s shortname from <current> to <new>
git remote remove <remote>        <--remove a remote

git clone <url> <where to clone>  <-- clone remote repository to given location
git clone ../remote_repo.git .    <--cloning example

If you clone a repository, the command automatically adds that remote repository under the name "origin". So, "git fetch origin" fetches any new work that has been pushed to that server since you cloned (or last fetched from) it. (git fetch will not do merge, just download the data)

If your current branch is set up to track a remote branch, you can use the git pull command to automatically fetch and then merge that remote branch into your current branch. By default, the git clone command automatically sets up your local master branch to track the remote master branch on the server you cloned from. Running git pull fetches data from the server you originally cloned from and automatically tries to merge it into the code you’re currently working on.

git fetch <remote>                <--get data from a remote project, it pulls down all the data from that remote project
git pull                          <--it is a combination of 2 comands: git fetch + git merge
git pull origin master            <--pulls any changes since the last time we have pulled from a repository

git push origin master            <-- push your master branch to your origin server
git push <remote> <branch>        <--submit branch to the remote repository
git push -u origin <branch name>  <--- -u is doing association between origin and our branch

----------------------------------------------

git diff <file>                             <--shows what exactly has been changed in given file
git diff <hash1> <hash2>                    <--show difference between hashes
git diff <source_branch> <target_branch>    <--doing a preview before a merge
git revert                                  <--will revert everything ???

git tag                                     <--showing tags (these are the versions which are added to  commits)
git tag -a v1.0 -m "Version 1.0 release"    <--point to the current release

----------------------------------------------

Fixing (undoing) a commit (git commit --amend):
(Committing accidentally too early and possibly forget to add some files, or we mess up our commit message.)

If we did a commit and then realize we forgot to stage the changes in a file we wanted to add to this commit, then we can use "git commit --amend" to fix this:
1. git commit -m "<initial commit>"
2. git add <forgotten file>
3. git commit --amend

The second commit replaces the first, as a result we end up with a single commit, and the earlier commit will not show up in the repository history

This command takes the staging area and uses it for the commit. If there are no changes since the last commit (if we run this command immediately after a commit) then only the commit message can be changed. A commit-message editor pops up, which already contains the message of our previous commit, which we can change and save.

-----------------------------------------------------

Branch and Merge:

Branch:
1. git branch                      <--shows how many branches we have (* shows which branch we are currently on)
2. git branch <name>               <--create a new branch
3. git checkout <branch>           <--switch to the given branch
4. do some work there

Merging a branch to master (locally and remotely)
(All merging is done into the branch where you are.)

1. git checkout master             <--switch to our master branch
2. git pull origin master          <--pull the changes down to make sure if there were any changes by others we have those
3. git branch --merged             <--lists the branches that have been merged already (our branch is not on the list)
4. git merge <branch name>         <--merge given branch to where we are (currently it is master)

After merging branch to the master (local), these changes can be pushed to the remote master:
git push origin master             <--all these changes are pushed to the master on the remote repository

-----------------------------------------------------

Deleting a branch

After we finished merging our branch, that feature is done and we can delete that branch:
1. git branch --merged             <--just to double check everything was successfully merged
2. git branch -d <branch>          <--deletes branch locally

If we pushed that branch to the remote repository, we can delete from there as well:
git branch -a                      <--list branches (our local branch is not there anymore, but we still have on remote repository)
git push origin --delete <branch>  <--to delete a branch on remote repository

-----------------------------------------------------

Fixing an accidental commit to a wrong branch:
(moving a commit to another branch)

1. git log                         <--check the hash of the commit you want to move (only first 6-8 characters)
2. git checkout <branch>           <--switch to the branch (you can check git log again)
3. git cherry-pick <hash>          <--it will bring over the given commit to the branch where I am

We still have the commit on the master branch (cherry-pick will not delete that). To delete that from the master branch:
4. git checkout master             <--switch to master branch
5. git log                         <--check hash and commits
6. git reset ….                    <--reset back a specified commit, it has 3 types (soft, mixed, hard):
        git reset --soft <hash>    <--reset back a commit, but will keep files in staging area
        git reset --mixed <hash>   <--(default) it is same as soft, but files will be not in staging area, instead in the working dir
        git reset --hard <hash>    <--reset all tracked file to the state that they were (it leaves any untracked file there)
7. git clean -df                   <--after git reset, get rid of any untracked directories (-d) and untracked file (-f)

-----------------------------------------------------

DEVOPS - RH 6,7 INSTALL

Red Hat - Netboot and Install from NIM server

A NIM server can boot up and install AIX servers from network using TFTP, BOOTP protocols. As other operating systems are using these same protocols for network boot (and install), our NIM server could be utilized for network booting and installing RHEL 6 or 7 servers as well. The process is basically the same on each environment, but the setup and configuration is different for AIX, RHEL 6 and RHEL 7. (In the Linux world there are many other methods to achieve this, like a Kickstart server, here wanted to show that NIM server is capable of doing these tasks.)

RHEL 6:
After network boot is started (tftp, bootp) the first reference on NIM is the /etc/yaboot.conf file, which points to a kickstart file (ks.cfg), which does the automatic install. For initiating a network boot we will use a fixed IP (172.23.72.15), but later during kickstart we will use dhcp (as it did not work with static ip, just with a local cdrom install) ks.cfg is configured to download install packages by HTTP (I tested with NFS and FTP but it did not really worked.)

RHEL 7:
Similar as above, just after network boot, /boot/grub/powerpc-ieee1275/grub.cfg file is used, this points to the kickstart config file as well.


Some additional details found on internet:
RHEL7 has moved to Grub2 as the boot loader, so there are a few things that are different... The biggest thing is that Grub2 will look for it's configuration in /boot on the TFTP server (not /etc like Yaboot does). RHEL7 is also smarter on NFS versions, so you shouldn't have to specify the version if using version 3 like you do on RHEL 6.

The Yaboot binary automatically looks for /etc in the TFTP directory. Since AIX doesn't chroot TFTP, you have to put things were Yaboot can find it. Similarly, if you are using Grub2 from and AIX boot system, it will look in /boot/grub2 for things it needs when network booting.


These steps will show how to install RH6 and RH7 servers via network from NIM:
1. HTTP access
2. Copy data from DVD
3. /etc/bootptab
4. Boot config files
5. /etc/tftpaccess.ctl
6. Kickstart file
7. Netboot

All the configuration will be done on the NIM server. A RH LPAR has been created earlier (same LPAR can be used for RH6 or RH7 install) and a fixed IP is reserved for this purpose:
172.23.72.15  - IP configured initially on RH for netboot
172.23.74.4   - IP of NIM server
172.23.72.1   - gateway
255.255.248.0 - subnet mask

----------------------------------------

1. HTTP access:

During installation we will use HTTP, so on our NIM server a webserver has been installed (Apache) and in httpd.conf this was needed to add at the end:

# vi /opt/freeware/etc/httpd/conf/httpd.conf
#RHEL image creation
Alias /export  "/export/"
<Directory "/export/">
    Options Indexes FollowSymLinks Includes MultiViews
    Require all granted
</Directory>


After that stop/start:
# /etc/rc.d/init.d/httpd start
# /etc/rc.d/init.d/httpd stop

----------------------------------------

2. Copy data from DVD

On the NIM server we need to copy the whole content of the RH DVD iso to a direrctory:
(this directory will be used during OS installation to install all the packages via http)

# loopmount….
# cp -prh /mnt /export/nim/rhel_6.10_ppc64
# cp -prh /mnt /export/nim/rhel_7.6_ppc64


For network boot we need to copy additional files from DVD, but there is a difference between RH6 and RH7, what files are needed on which locations:

RHEL 6:
- copy yaboot.conf from iso: /ppc/ppc64/yaboot.conf to /etc/yaboot.conf on NIM
- copy vmlinux (linux kernel) and initrd.img under /tftpboot directory
# mkdir -p /tftpboot/rhel6_ppc64
# cp /export/nim/rhel_6.10_ppc64/ppc/ppc64/initrd.img /tftpboot/rhel6_ppc64
# cp /export/nim/rhel_6.10_ppc64/ppc/ppc64/vmlinuz /tftpboot/rhel6_ppc64


RHEL 7:
the main directory is /boot,  which was copied from ISO image:
# cp -prh /export/nim/rhel_7.6_ppc64/boot /boot

Beside that initrd.img and vlinuz was also copied to /boot:
# cp /export/nim/rhel_7.6_ppc64/ppc/ppc6/initrd.img /boot
# cp /export/nim/rhel_7.6_ppc64/ppc/ppc6/vmlinuz /boot

----------------------------------------

3. /etc/bootptab

/etc/bootptab has to be updated manually so network communication is possible during netboot.

Added these lines at the end of bootptab file:
(yaboot and core.elf files have been copied earlier)

RHEL 6:
ls-rh-ppc64-base:bf=/export/nim/rhel_6.10_ppc64/ppc/chrp/yaboot:ip=172.23.72.15:ht=ethernet:sa=172.23.74.4:gw=172.23.72.1:sm=255.255.248.0:

RHEL 7:
ls-rh-ppc64-base:bf=/boot/grub/powerpc-ieee1275/core.elf:ip=172.23.72.15:ht=ethernet:sa=172.23.74.4:gw=172.23.72.1:sm=255.255.248.0:


After installation completed these lines should be removed manually.

----------------------------------------

4. Boot config files

The main config file for inital network boot is different for RHEL6 and 7:
RH6: yaboot.conf
RH7: grub.cfg

RHEL 6: /etc/yaboot.conf
During network boot a filename with MAC address is searched, if that file is missing, then boot process will check different MAC address and IP variations. For each variation there is a timeout if that file is not found… so we don’t want to wait 2 mins, create a link with MAC address to this file:
# cd /etc
# ln -s yaboot.conf 01-fa-8e-09-73-18-20

(I tried to use NFS which worked here, but later in the kickstart file it did not work NFS, so I changed to HTTP everywhere)
# cat /etc/yaboot.conf
init-message = "\nWelcome to the 64-bit Red Hat Enterprise Linux 6.10 installer!\nHit <TAB> for boot options.\n\n"
timeout=50
default=linux

image=rhel6_ppc64/vmlinuz
        label=linux
        initrd=rhel6_ppc64/initrd.img
#       append="ks=nfs:nfsvers=3:172.23.74.4:/export/nim/misc/rhel6_ppc64_ks.cfg ksdevice=eth0 ip=172.23.72.15 netmask=255.255.248.0 gateway=172.23.72.1"
        append="ks=http://172.23.74.4/export/nim/misc/rhel6_ppc64_ks.cfg ksdevice=eth0 ip=172.23.72.15 netmask=255.255.248.0 gateway=172.23.72.1"
        read-only



RHEL 7: /boot/grub/powerpc-ieee1275/grub.cfg
# cat /boot/grub/powerpc-ieee1275/grub.cfg
set default=0
set timeout=5

echo -e "\nWelcome to the Red Hat Enterprise Linux 7.6 installer!\n\n"

menuentry "RHEL for PowerPC" {
     linux /boot/vmlinuz ro ip=172.23.72.15::172.23.72.1:255.255.248.0:ls-rh-ppc64-base:eth0:none inst.repo=http://172.23.74.4/export/nim/rhel_7.6_ppc64/ inst.ks=http://172.23.74.4/export/nim/misc/rhel7_ppc64_ks.cfg
     initrd /boot/initrd.img
}

----------------------------------------

5. /etc/tftpaccess.ctl

On the NIM server we need to grant access to the needed resources. Some lines have been added to /etc/tftpaccess.ctl.

# cat /etc/tftpaccess.ctl
# NIM access for network boot
allow:/tftpboot
allow:/tftpboot/rhel6_ppc64
allow:/export/nim/rhel_6.10_ppc64
allow:/etc/yaboot.conf


Some side note:
NFS export is not needed as we use HTTP. I did some experiment with NFS and I leave here commands I used (just in case):
exportfs -i /export/nim/rhel_6.10_ppc64
exportfs -i /export/nim/misc

unexport:
exportfs -u /export/nim/rhel_6.10_ppc64
exportfs -u /export/nim/misc

----------------------------------------

6. Kickstart file:

Kickstart files are controlling what settings should be used during install (timezone, language...) 2 files have been created in /export/nim/misc, one for RH6 and one for RH7. (the url line contains the path on NIM sever for the install resources)

RHEL 6:
# cat rhel6_ppc64_ks.cfg
lang en_US
keyboard us
timezone Europe/Vienna --isUtc
rootpw $1$3Qm1F030$Q6ExOTiF/ndBk7neDKNZp1 --iscrypted
#platform IBM pSeries
reboot
text
#cdrom
url --url http://aix-mgmt.mydomain.org/export/nim/rhel_6.10_ppc64
bootloader --location=mbr --append="rhgb quiet crashkernel=auto"
zerombr
clearpart --all --initlabel
autopart
#network --device=eth0 --bootproto=static --ip=172.23.72.15 --netmask=255.255.248.0 --gateway=172.23.72.1
network --device eth0 --bootproto dhcp --hostname ls-rh-ppc64-base
auth --passalgo=sha512 --useshadow
selinux --disabled
firewall --disabled
skipx
firstboot --disable

%post
mkdir /root/.ssh
echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkG2Q00DnTLjXrU+gP7mxdrEMao55555555555Xs2HzfIjX73f7imMO5dEP7lWEFuKbXgTjgMsaWnFC6SCRiqCBFi9aPcTfTc12FVUHf2D18oqoi2LjqAslHSFUILRwhJi0dn0k6u8U1k7c7oV3VWb1bsMipn86/De+axa7endkXfTLOgWu3A1c/2H/Wf2nfpw2ElPOPZJJ3kVGqCJXFnHbOrWm9nw1GCDfQPNi82qySiuuCPBzjb953JdMrNVX++tmObFr6veH9775z4ucE/a67wp/XnGEy2lYogQcWj5lPULc6bUFSQvl0hT1HotvQlaywm2fB1eb6U88RSUV5VZ user@aix.mydomain.org' >> /root/.ssh/authorized_keys
%end

RHEL 7:
same as for RHEL 6, only the url line is different:
url --url http://aix-mgmt.lnz.lab.dynatrace.org/export/nim/rhel_7.6_ppc64

----------------------------------------

7. Netboot:

After all these steps have been prepared, netboot of RH LPAR can be initiated from HMC:
ssh hscroot@hmc01 "lpar_netboot -v -f -D -i -t ent -s auto -T off -d auto -S 172.23.74.4 -C 172.23.72.15 -G 172.23.72.1 -K 255.255.248.0 rh-ppc64-lpar default_profile man-sys-01"

----------------------------------------

if needed logging in yaboot.conf or grub.conf: https://wwoods.fedorapeople.org/doc/boot-options.html#_inst_syslog
user@aix-mgmt:/var/log $ tail -f user

----------------------------------------

DEVOPS - YUM

YUM (Yellowdog Updater Modified)
https://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/ezinstall/ppc/README-yum

YUM is package manager for RPM packages, which allows automatic updates, package and dependency management. Yellowdog was a Linux distribution which was based on Red Hat. It had a package manager, which was called Yellowdog UPdater (YUP). It has been rewritten to support Red Hat based Linux system, and since then it is called YUM (Yellowdog Updater Modified.) YUM under the hood depends on RPM; which is a packaging standard.

YUM doesn't recognize AIX installp dependencies. For example: OpenSSL libraries are provided by AIX in installp format, so if an RPM package depends on a specific OpenSSL version then user should make sure to keep his OpenSSL version updated.

--------------------------------------------------------------------------------

RPM:

It is a free and open source package management system, which is available on AIX.

Without YUM thes RPM commands can be used on AIX
rpm -qa                       shows what rpm packages are installed
rpm -ql <package name>        shows where the files are installed (rpm -qlp .. shows the absolut paths???)
rpm -q --filesbypkg cdrecord  list all the files on installed rpm package
rpm -qf /usr/bin/lynx         query a file to find the source rpm package
rpm -qi <package name>        list information on an installed rpm package
rpm -qR <package name>        list all dependencies on any rpm package
     
rpm -ivh httpd-2.2.8.aix5.1.rpm install the rpm package

rpm -ivh --force *.rpm
rpm -ivh --force --nodeps <package name>    does not check dependency (same can be done with "rpm -Uvh..." for upgrade)
rpm -e <package name>         removes the rpm package

rpm -Va                       shows which files are missing from the RPM database
rpm -Vv <package name>        verifies a package
rpm --rebuilddb               compress and rebuild the RPM database

/usr/sbin/updtvpkg            enables the rpm command to recognize that the libraries have been installed

In some cases you might get an error about failed dependencies when you install RPMs on AIX (for example, error: failed dependencies: libX11.a(shr4.o) is needed by tk-8.3.3-1). Most likely the error occurs because the rpm command does not recognize the shared library. If the error occurs, check to see if the X11 libraries are installed in the directory /usr/lpp/X11/lib. If they are not installed, use the AIX product media to install them. After you have installed the libraries, run the above command (updtvpkg). The command enables the rpm command to recognize that the libraries have been installed.

---------------------------------

Installing yum on AIX

1. install latest rpm.rte: https://ftp.software.ibm.com/aix/freeSoftware/aixtoolbox/INSTALLP/ppc/
# installp -qacXYd rpm.rte all

2. install all rpm packages contained in yum_bundle.tar
https://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/ezinstall/ppc/
# tar -xvf yum_bundle.tar
# rpm -Uvh *

Another way to install yum is to download yum.sh from https://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/ezinstall/ppc/.
Running this script will download and install the latest rpm.rte and yum_bundle.tar.

yum.conf file will be installed under the path /opt/freeware/etc/yum.
Additional yum repository files can be created under the path /opt/freeware/etc/yum/repos.d.

---------------------------------

AIX Toolbox Repositories

IBM provides the most widely used GNU and opensource Linux tools on AIX (like gcc, git, coreutils, perl, python, ruby, php etc.). These can be downloaded from the AIX Toolbox site: https://www.ibm.com/support/pages/node/882892

It is also possible to use AIX Toolbox as a yum repository, so dependencies will be handled automatically by yum. After installing yum, by default 3 IBM repositories are enabled for rpm packages in /opt/freeware/etc/yum/yum.conf.

These 3 repositories are: ppc, noarch and ppc-X.X
ppc:     Contains most of the rpms. These rpms are built on AIX 5.3, 6.1 and these can be installed on any higher versions of AIX
noarch:  Contains architecture independent packages (scripts, text files, which are not binaries), these can be installed on any AIX
ppc-6.1: Contains gcc which can be installed only on aix6.1.
ppc-7.1: Contains gcc which can be installed only on aix7.1.
ppc-7.2: Contains gcc/gcc-go which can be installed only on aix7.2.

The reason behind this is, that some of the rpm packages like gcc can only be installed on a specific version of AIX. For example, gcc built on AIX 6.1 will be allowed to be installed only on AIX 6.1 machines, so with the configuration of these separate repositories "yum install gcc" will work on all AIX servers, irrespective of AIX version.

-------------------------------------------------

/opt/freeware/etc/yum/yum.conf                    <--main yum configuration file

yum clean all                                     <--cleans up cache (to see new config, packages in repo cache has to be cleaned up)
yum search <keyword>                              <--to find packages containing the specified keywordin description from repository
yum list <package>                                <--shows status of a package (if it is installed or available in repositiry)
yum list <package> --showduplicates               <--lists all versions in repository (installed and all available versions)
yum info <package>                                <--info about a package
yum whatprovides <filename>                       <--shows which package contains given file (like: yum whatprovides zlib.so)
yum provides <filename>                           <--to find which package provides the file

yum install <package>                             <--installs a package + dependencies (more packages: yum install package1 package2 …)
yum install <package_name>-<version_info>         <--installs a specific version (like: yum install gcc-6.3.0-1)
yum localinstall </path/to/package>               <--installing a package from local path instead of a repository

yum update <package>                              <--updates a package (with its dependencies if needed)
yum check-update                                  <--check if a newer version of a package is available
yum remove <package>                              <--remove a package

yum history                                       <--lists history of yum actions (same as yum history list all)
yum history info <transaction_ID>                 <--gives details about the specified history transaction id
yum history undo <transaction_ID>                 <--roll backs the given tranaction id

yum repolist                                      <--list repositories
yum --disablerepo="*" --enablerepo="epel" list available       <--lists packages only in a specific repo (use output of "yum repolist" )
yum --disablerepo=* --enablerepo=LIVE* list Centrify*          <--lists installed and available packages from LIVE* repos

createrepo --checksum sha --update /etc/repo      <--update repo after a new package is copied there

---------------------------------

Creating local YUM repository

We need to choose an AIX or Linux server with HTTP access (Apache/NGNIX) that can be used as a repository server. The ideal scene is if it has access to the internet (so it can sync packages from IBM site). If it is not possible then we need to copy there manually the rpm packages. The YUM clients are connecting to the repository server using HTTP connections.  (I guess that is why the curl package is a prerequisite for the yum package.)

Detailed steps are described here:  https://developer.ibm.com/articles/configure-yum-on-aix/ and here: https://www.djouxtech.net/posts/aix-yum-installation/

1. Install yum-utils and createrepo (with dependencies)

2. Download all rpm packages what you need in repo
We can use the default IBM repositories to download rpm packages (if we have internet on the repository server). By default /opt/freeware/etc/yum/yum.conf contains the IBM repositories (yum repolist should show them), and reposync command will sync (download) content of a repository
# reposync -p <target_path> -r AIX_Toolbox -a ppc
# reposync -p <target_path> -r AIX_Toolbox_61 -a ppc
# reposync -p <target_path> -r AIX_Toolbox_71 -a ppc
# reposync -p <target_path> -r AIX_Toolbox_72 -a ppc
# reposync -p <target_path> -r AIX_Toolbox_noarch
(-p path to download, -r: the name of the repository, -a: architecture)

3. create repo
Run createrepo for all the downloaded packages:
# createrepo <target_path>/AIX_Toolbox
# createrepo <target_path>/AIX_Toolbox_61
# createrepo <target_path>/AIX_Toolbox_71
# createrepo <target_path>/AIX_Toolbox_72
# createrepo <target_path>/AIX_Toolbox_noarch

4. on AIX servers update repo config files with correct locations
On client servers, create new repo file in /opt/freeware/etc/yum/repos.d: like localrepos.repo:
# cat /opt/freeware/etc/yum/repos.d/localrepos.repo
[local_AIX_Toolbox]
name=local AIX generic repository
baseurl=http://reposerver.mydomain.com/ias-AIX/Toolbox/
enabled=1
gpgcheck=0
priority=1

[local_AIX_Toolbox_noarch]
name=local AIX noarch repository
baseurl=http://reposerver.mydomain.com/ias-AIX/Toolbox_noarch/
enabled=1
gpgcheck=0
priority=1

[locaL_AIX_Toolbox_71]
name=local AIX 7.1 specific repository
baseurl=http://reposerver.mydomain.com/ias-AIX/Toolbox_71/
enabled=1
gpgcheck=0
priority=1

5. disable IBM repos (if needed)
If localrepos will be used in /opt/freeware/etc/yum/yum.conf file IBM repositories can be disabled (enabled=0)

-------------------------------------------------

YUM repository update:

If we want to add additional packages to a repo, do these steps on the repository server:
1. copy package to correct repository (make sure permissions and user are correct)
3. update repo: createrepo --checksum sha --update /srv/packer/ias-AIX/Toolbox
4. on clients clean cache: yum clean all (so new packages vill be visible for yum commands)

-------------------------------------------------

DEVOPS - QEMU

QEMU - AIX on x86

QEMU (Quick EMUlator) is a machine emulator program, which is capable to create VMs with different processor architectures. For example if we want to test a Solaris (with RISC processor) or an HP-UX (with HPPA) or an AIX (with Power processor) QEMU can emulate that processor type and run the specified OS on that. So on Windows or Linux (on x86) with QEMU we can run AIX.

After installing QEMU, we can use a command to create an empty virtual disk and after a VM (with specified RAM and additional devices). If we use an AIX DVD during the boot, AIX will be installed there. (After installation not all AIX commands are available but many things still possible to do.)

Without any prior experience in QEMU, I used these 2 sources:
https://astr0baby.wordpress.com/2018/11/04/running-aix-7-2-tl3sp1-on-x86_64-via-qemu-system-ppc64/
http://gibsonnet.net/blog/cgaix/resource/AIX_QEMU_blog.pdf

--------------------------------------------------------------

AIX on Linux

Very important: be patient during this process, things can be (very) slow sometimes.

On Windows I installed VirtualBox and I created a Linux VM (CentOS 8). The plan was that on this linux VM in VirtualBox I will create an AIX VM using QEMU. After CentOS 8 installation completed, I configured network, so I could ssh to my linux server and do these steps:

0. Download AIX DVD 
From IBM ESS site download AIX DVD. I used 7.1 TL5 SP5, DVD1 is enough (710505.iso). Copy  to any directory, I used /root/aix.

1. Install Qemu
I could not find latest qemu package on linux (only version 2 was available, but that did not support latest Power systems), so it had to be compiled from source code:
(I had to install git and other missing things like glib-devel ...)
# git clone git://git.qemu.org/qemu.git   <--it downloads latest source code (automatically creates a qemu dir where I start command)
# cd qemu                                 <--go to qemu dir which was created by git
# mkdir build; cd build                   <--create a build directory and go there
# ../configure                            <--checks if all requirements are OK
# make                                    <--this compiles the source code (it can take long)
# make install                            <--it installs compiled software
# qemu-system-ppc64 --version             <--check if it works correctly
QEMU emulator version 4.2.50 (v4.2.0-2665-g3d0ac34603)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

2. Create a disk file
Go to the AIX DVD dir and create a disk which will be used by AIX
# qemu-img create -f qcow2 hdisk0.qcow2 20G
It will create an empty 20GB qcow2 disk image file, but its initial size is very small, as it is a sparse file. (So we have AIX DVD and this image file in our dir)

3. Create an AIX VM
This command will create an AIX VM with specified settings and boot it from DVD:
# qemu-system-ppc64 -cpu POWER8 -machine pseries -m 2048 -serial stdio -drive file=hdisk0.qcow2,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom 710505.iso -prom-env "boot-command=boot cdrom:"

Parameters used:
-cpu: processor type, here POWER8
-machine: machine type, here pseries
-m: memory size, here 2GB
-serial: redirects serial output to specified device
-drive: the disk file we created before
-device: virtio-scsi-pci (not fully sure, but maybe this will create a virt. eth. adapter device)
-device: scs-hd (probably this will create hdisk device)
-cdrom: this is our downloaded AIX DVD which will be used during boot
-prom-env: sets NVRAM variable for PPC and SPARC servers, here a boot-command is used, which points to cdrom (DVD)

AIX boot will take several minutes, sometimes output can hang for 3-5 minutes. If everything is fine, the usual AIX install menu will apper (choose console and english language and default settings during install). The install will take about 2 hours, and disk file size will grow to 2.3GB. The installation process tries to restart AIX automatically after the install completed, but it was hanging, so after some time I did CTRL-C.

4. Fix fsck64 issue in Maintenance mode
fsck fails during boot and boot will hang. A workaround to this issue is to overwrite fsck64 file with exit 0. Boot our new AIX to maintenance moe:
# qemu-system-ppc64 -cpu POWER8 -machine pseries -m 2048 -serial stdio -drive file=hdisk0.qcow2,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom 710505.iso -prom-env "boot-command=boot cdrom:"

When the "Welcome to Base Operating System" menu appears:  choose 3 (Start Maintenance Mode) --> 1 (Access a Root VG) -->  0 Continue  --> 1 (VG on hdisk0) --> 1 (Access this VG and start a shell). 

After that we will get a prompt with an environment where filesystems are already mounted:
# cd /sbin/helpers/jfs2    <--go to this location
# > fsck64                 <--delete content of that file
# vi fsck64                <--vi and insert these 2 lines. File is readonly, so save and quit with :w! and :q
#!/bin/ksh
exit 0

# cat fsck64               <--it should contain 2 lines (#!/bin/ksh and exit 0)
# sync ; sync              <--write cached memory to disk
# halt                     <--halt AIX


5. Boot from disk
This is the first time AIX will be booted from disk:
# qemu-system-ppc64 -cpu POWER8 -machine pseries -m 2048 -serial stdio -drive file=hdisk0.qcow2,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom 710505.iso -prom-env "boot-command=boot disk:"

This boot will take much longer (about 5-10 minutes) as some daemons will fail and it will hang at NFS part until timeout, but we will get a prompt. After that some services will be disabled (so next boots will be faster) and installed ssh from DVD.
Disabling services in inittab:
# rmitab rcnfs
# rmitab cron
# rmitab piobe
# rmitab qdaemon
# rmitab writesrv
# rmitab naudio2
# rmitab naudio
# rmitab aso
# rmitab clcomd
# chrctcp -S -d tftpd

Install ssh and change root pw:
# chfs -a size=+200M /home
# mount -v cdrfs -o ro /dev/cd0 /mnt
# mkdir /home/ssh; cd /mnt/installp/ppc; cp openssh.base openssh.license openssh.man.en_US openssh.msg.en_US /home/ssh; umount /mnt
# cd /home/ssh; installp -acXY -d . all
# passwd root
# halt

After that booting up from disk should be faster.

6. Network setup
I wanted to use ssh on AIX, so network setup was needed. (When I worked in qemu terminal and I did ctrl-c, the whole qemu session (process) was stopped, so I had to boot again the VM, that was another reason to use ssh sessions.) Without knowing linux network virtualization techniques, it was not easy, but found this site, which worked: http://hostrepo.com/article.php?id=193. Basically a network bridge was needed, so network communication could be possible between my Linux and AIX VM. On Linux this network bridge function can be achieved by a TAP (Terminal Access Point) device and with ARP proxy. The following steps are needed each time after Linux is restarted, so these can be put in a script as well. (There are lots of internet sites which shows different other methods.)

current IP on Linux: 10.0.2.15 (enp0s3)
planned IP on AIX: 10.0.2.16 (en0)

On the Linux VM:
# ip tuntap add tap0 mode tap                      <--create tap0 interface ('ifconfig' will not show it, 'ip a' shows state is DOWN)
# ip link set tap0 up                              <--bring up tap0 ineterface ('ifconfig' will show tap0 state is UP)
# echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp  <--enable proxy arp on tap0 (empty file was already there)
# ip route add 10.0.2.16 dev tap0                  <--setup routing for 10.0.2.16 (AIX IP) on tap0 device ('ip route' shows new route)
# arp -Ds 10.0.2.16 enp0s3 pub                     <--broadcast ARP for AIX IP ('yum install net-tools' was needed for arp, netstat commands)

Boot up AIX using tap0 device:
# cd /root/aix
# qemu-system-ppc64 -cpu POWER8 -machine pseries -m 2048 -serial stdio -drive file=hdisk0.qcow2,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom 710505.iso -prom-env "boot-command=boot disk:" -net nic,macaddr=56:44:45:30:31:32 -net tap,script=no,ifname=tap0

Then on AIX:
# chdev -l en0 -a netaddr=10.0.2.16 -a netmask=255.255.255.0 -a state=up

After that ping and ssh is possible to AIX. AIX IP configuration will survive restarts, but Linux steps are needed each time after Linux VM is rebooted. In future if we want to start AIX in the background without a console this can be used:
# qemu-system-ppc64 -cpu POWER8 -machine pseries -m 2048 -drive file=hdisk0.qcow2,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom 710505.iso -prom-env "boot-command=boot disk:" -net nic,macaddr=56:44:45:30:31:32 -net tap,script=no,ifname=tap0 -daemonize

(logout from session, after AIX shutdown is possible using "~~.", same as in HMC console)

--------------------------------------------------------------

AIX on Windows

It is possible to do the same steps (as above) directly on Windows without VirtualBox and a Linux VM. The only difference is that we need to give qemu commands in command prompt (cmd) or in a terminal emulator (like MobaXterm). With these terminals on Windows there could be character problems.

We need to install latest Qemu on Windows: https://www.qemu.org/download/#windows. After install completed, above steps can be  followed, with slight modification in qemu commands:

Create a disk file: 
"C:\Program Files\qemu\qemu-img.exe" create -f qcow2 hdisk0.qcow2 20G

Boot an AIX VM and install AIX from DVD:
"C:\Program Files\qemu\qemu-system-ppc64.exe" -cpu POWER8 -machine pseries -m 2048 -serial stdio -drive file=hdisk0.qcow2,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom 710505.iso -prom-env "boot-command=dev / 0 0 s\" ibm,aix-diagnostics\" property boot cdrom:\ppc\chrp\bootfile.exe -s verbose" -display vnc=:1

Boot our newly installed VM into Maintenance Mode and fix fsck64:
"C:\Program Files\qemu\qemu-system-ppc64.exe" -cpu POWER8 -machine pseries -m 2048 -serial stdio -drive file=hdisk0.qcow2,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom 710505.iso -prom-env "boot-command=boot cdrom:" -display vnc=:1

Boot AIX from disk:
"C:\Program Files\qemu\qemu-system-ppc64.exe" -cpu POWER8 -machine pseries -m 2048 -serial stdio -drive file=hdisk0.qcow2,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom 710505.iso -prom-env "boot-command=boot disk:" -display vnc=:1

--------------------------------------------------------------

DEVOPS - PUPPET FACTER

Facter

In Puppet, facter is a standalone tool that holds the environment level variables. In can be considered similar to env variable. In Puppet, the key-value pair is known as “fact”.  We can list what Facter knows about the system by using command: facter

# facter

...

...

facterversion => 3.14.9

filesystems => ext2,ext3,ext4,xfs

fips_enabled => false

hypervisors => {

  virtualbox => {

    revision => "136177",

    version => "6.1.4"

  }

is_virtual => true

kernel => Linux

kernelmajversion => 4.18

kernelrelease => 4.18.0-147.el8.x86_64

kernelversion => 4.18.0

...

...

Facter command can be used to list all the different environment variables and its associated values. These collection of facts comes with facter out-of-the-box and are referred to as core facts. One can add custom facts to the collection. Facter and facts are available throughout Puppet code as “global variable”, which means it can be used in the code at any point of time without any other reference.

file {'/tmp/userfile.txt': 

      ensure => file, 

      content => "the value for the 'OperatingSystem' fact is: $OperatingSystem \n", 

   } 

file{'motd':

  ensure  => file,

  path    => '/etc/motd',

  mode    => '0644',

  content => "IP address is ${ipaddress}, hostname is ${fqdn}. It is running ${operatingsystem} ${operatingsystemrelease} and Puppet ${puppetversion}",

}

===========================================

There are three types of fact that can be used and defined: Core Facts, Custom Facts, External Facts

Core Facts

Core facts are defined at the top level and accessible to all at any point in the code.

Just before an agent requests for a catalog from the master, the agent first compiles a complete list of information available in itself in the form of a key value pair. Facter data is treated as global variable. The facts are then available as top level variable and the Puppet master can use them to compile the Puppet catalog for the requesting agent. Facters are called in manifest as normal variable with $ prefix.

if ($OperatingSystem == "Linux") { 

   $message = "This machine OS is of the type $OperatingSystem \n" 

} else { 

   $message = "This machine is unknown \n" 

} 

file { "/tmp/machineOperatingSystem.txt": 

   ensure => file, 

   content => "$message" 

[root@puppetagent1 /]# facter OperatingSystem 

Linux  

[root@puppetagent1 /]# puppet apply /tmp/ostype.pp 

Notice: Compiled catalog for puppetagent1.codingbee.dyndns.org 

in environment production in 0.07 seconds 

Notice: /Stage[main]/Main/File[/tmp/machineOperatingSystem.txt]/ensure: 

defined content as '{md5}f59dc5797d5402b1122c28c6da54d073' 

Notice: Finished catalog run in 0.04 seconds  

[root@puppetagent1 /]# cat /tmp/machinetype.txt 

This machine OS is of the type Linux

Custom Facts

Using the “export FACTER” Syntax one can manually add the facts using the export FACTER_{fact’s name} syntax.

Example:

[root@puppetagent1 facter]# export FACTER_tallest_mountain="Everest" 

[root@puppetagent1 facter]# facter tallest_mountain Everest

External Fact

On the agent machine, we need to create a directory as mentioned below.

$ mkdir -p /etc/facter/facts.d

Create a Shell script in the directory with the following content.

$ ls -l /etc/facter/facts.d 

total 4 

-rwxrwxrwx. 1 root root 65 Sep 18 13:11 external-factstest.sh 

$ cat /etc/facter/facts.d/external-factstest.sh 

#!/bin/bash 

echo "hostgroup = dev" 

echo "environment = development"

Change the permission of the script file.

$ chmod u+x /etc/facter/facts.d/external-facts.sh

Once done, we can now see the variable present with the key/value pair.

$ facter hostgroup 

dev 

$ facter environment 

development 

Hiera

Hiera is a "Hierarchical Database" to store values for variables (key/value pair). It provides to Puppet a separation between the code and the data. With Hiera it is possible to write Puppet code, where the value for a variable will be searched in Hiera database and if it is found there our Puppet class will use that. Puppet classes can request whatever data they need, and your Hiera data will act like a site-wide config file. It makes our code easy to share and reuse, because the code and the data is separated.

Without Hiera our class would look like this:
class puppet::params {
 $puppetserver = "puppet.example.com"
}

Here we have a variable that will be needed somewhere else, and Hiera is a good place to keep these kind of variables. Hiera presents itself to Puppet as a function call and searches in special YAML or JSON files for the given variable. As an example one of our Hiera data file (written in YAML) could look like this:
puppetserver = 'puppet.example.com'

Then we can rewrite our puppet code like this:
class puppet::params {
 $puppetserver = hiera('puppetserver')
}

In this example Puppet will use the hiera function to get the string value 'puppet.example.com' and place it into the $puppetserver variable.

Hiera helps us to separate configuration from data. It helps us to create modules that are interchangeable blocks, so  the details of that configuration (the data) stays in Hiera data files and the logic of the module stays in the Puppet manifests.

==============================

Hiera installation

Hiera is usually installed on  Puppet Master server, it is optional and unnecessary on agent nodes.
(When I installed on CentOS puppet agent ( dnf install puppet-agent), after that I could use hiera as well.)

puppet resource package hiera ensure=install  <--installing hiera with Puppet
gem install hiera                             <--installing hiera using ruby

After installation the main Hiera config file hiera.yaml is available. This file lists how we want to use for store key-value pairs (YAML or JSON) and lists as well all the hierarchy levels from top to down, where hiera will search for the requested variables. (Each top-level key in the hash must be a Ruby symbol with a colon (:)

An example hiera.yaml file
(it is written version 3, later versions have other syntax)

---                                           <--these 3 dashes (---) show the start of the document
:backends:                                    <--lists how we want to store key-value pairs (yaml, json...),
  - yaml
:yaml:                                        <--for yaml we can set some configuration settings (like datadir...)
  :datadir: /etc/puppet/hieradata             <--the directory in which to find yaml data source files
:hierarchy:                                   <--lists how search should happen, in which order hierarchy levels are followed
  - "%{::fqdn}"                               <--these are the name of the files where data is stored (in this case like myhost.domain.com.yaml)
  - "%{::custom_location}"
  - common                                    <--the file name here will be common.yaml (/etc/puppet/hieradata/common.yaml)


These files in the hierarchy are called data sources, and they can be:
Static data source:  A hierarchy element without any variables used there (without any interpolation tokens). A static data source will be the same for every node. In the example above, "common" is a static data source, because a virtual machine named web01 and a physical machine named db01 would both use common.

Dynamic data source: A hierarchy element with at least one interpolation token (variable). If two nodes have different values for the variables it references, a dynamic data source will use two different data sources for those nodes. In the example above: the special $::fqdn Puppet variable has a unique value for every node. A machine named web01.example.com would have a data source named web01.example.com.yaml, while a machine named db01.example.com would have db01.example.com.yaml.

==============================

Backends

A Backend is that part of a computer system or application that is not directly accessed by the user. It is typically responsible for storing and manipulating data. In Hiera the backends are those files where the actual key-value pairs are stored. Hiera will search in these files and provides the data for the user.

The 2 main types which can be used are yaml and json. (It is possible to use other backends or write our own backends.) For each listed backends, the datadir is specified. This is the directory where our yaml (or json) files are stored (where the data source files are stored). It is possible to use variables (like %{variable}) with datadir, for example: /etc/puppet/hieradata/%{::environment}, so we can keep our production and development data separate.

------------------
Multiple Backends
We can specify multiple backends as an array in hiera.yaml. Hiera will give priority to the first backend, and will check every level of the hierarchy in it before moving on to the second backend.
For example in the following yaml fiel we use yaml and json backends (in this order):
---
:backends:
  - yaml
  - json
:yaml:
  :datadir: /etc/puppet/hieradata
:json:
  :datadir: /etc/puppet/hieradata
:hierarchy:
  - one
  - two
  - three

If we search for something in the hierarchy, then hiera will check files in this order:
one.yaml
two.yaml
three.yaml
one.json
two.json
three.json
------------------
==============================

Hierarchies

Hiera uses an ordered hierarchy to look up data, and this hierarchy is written in the hiera.yaml file. Each element in the hierarchy must be a string, which may or may not include variables (interpolation tokens). Hiera will treat each element in the hierarchy as the name of a data source.

For example:
:hierarchy: 
  - "%{::fqdn}"
  - common

Hiera uses Puppet facts (like fqdn) and if we use these facts as variables in the hierarchy definitions, then we can create separate yaml files (server1.yaml, server2.yaml... based on fqdn) with their own separate configuration values for each server (server 1 needs this package, server2 needs other package). Remove Puppetʼs $ (dollar sign) prefix when using its variables in Hiera. (That is, a variable called $::clientcert in Puppet is called ::clientcert in Hiera.) Puppet variables can be accessed by their short name or qualified name

Each element in the hierarchy resolves to the name of a data source(myhost.example.com.yaml, common.yaml). Hiera will check these data sources in order, starting with the first. If a data source in the hierarchy doesnʼt exist (the yaml file was deleted), Hiera will move on to the next data source. If a data source exists but does not have the piece of data Hiera is searching for, it will move on to the next data source (first checks myhost.examle.com.yaml, if data is not found it will check common.yaml). If a value is found in a normal (priority) lookup, Hiera will stop and return that value.  If Hiera goes through the entire hierarchy without finding a value, it will use the default value if one was provided, or fail with an error.

For example here the numbers show which data source (yaml file) is searched in which order:
(it shows the hierarchy levels from hiera.yaml and the facts, which are used during search for node db01.example.com)

So the final hierarchy in this example:
1. db01.example.com.yaml
2. development.yaml
3. common.yaml


(There are other search mechanisms, for example when Hiera will not stop at the first occurencce, but searches through all the hierarchy levels, and at the end combines the different values into an array. It is called the Array merge lookup method. And there is another method called Hash merge, which I never used.)

==============================

Data Sources (YAML, JSON)

YAML
The yaml backend looks for data sources on disk, in the directory specified in its :datadir setting. It expects each data source to be a text file containing valid YAML data, with a file extension of
.yaml. No other file extension (e.g. .yml) is allowed.

yaml data format examples:

---
# array
apache-packages:
  - apache2
  - apache2-common
  - apache2-utils

# string
apache-service: apache2

# interpolated facter variable
hosts_entry: "sandbox.%{fqdn}"

# hash
sshd_settings:
root_allowed: "no"
password_allowed: "yes"

# alternate hash notation
sshd_settings: {root_allowed: "no", password_allowed: "yes"}

# to return "true" or "false"
sshd_settings: {root_allowed: no, password_allowed: yes

-------------------------------

JSON


The json backend looks for data sources on disk, in the directory specified in its :datadir setting. It expects each data source to be a text file containing valid JSON data, with a file extension of
.json. No other file extension is allowed.

json data format examples:

{
"apache-packages" : [
"apache2",
"apache2-common",
"apache2-utils"
],

"hosts_entry" : "sandbox.%{fqdn}",

"sshd_settings" : {
    "root_allowed" : "no",
    "password_allowed" : "no"
  }
}

===============================

Commands:

puppet lookup <variable>                       it will search for the given variable

hiera <variable>                               search for the given variable in hiera
   -c <yaml conf file>                         path to an alternate hiera.yaml file
   -d                                          debug mode

hiera my_var ::fqdn=localhost.localdomain      searching for variable (my_var) in hierarchy level, where ::fqdn is mentioned in hiera.yaml file
$gccs = hiera('gcc::versions', undef)          in puppet code a variable can get a value using hiera (if hiera does not find variable, it wll get undef value)


===============================

My test setup

hiera config file:
# cat hiera.yaml
---
:backends:
  - yaml
:yaml:
  :datadir: /root/hieradata
:hierarchy:
  - "node/%{::fqdn}"
  - "osfamily/%{osfamily}"
  - common

(I checked with facter what is current fqdn and it showd localhost.localdomain so I created that yaml file.)

directory and file structure:
/root/hieradata
├── node
│   └── localhost.localdomain.yaml
├── osfamily
│   ├── Debian.yaml
│   └── RedHat.yaml
└── common.yaml


content of yaml files:
# cat localhost.localdomain.yaml
my_var: node
gcc_version:
 - '6.4.0'
 - '8.3.0'
 - '9.1.0'

# cat Debian.yaml
"tools::working_dir" : "/opt/debian"
my_var: debian

# cat RedHat.yaml
"tools::working_dir" : "/opt/redhat"

# cat common.yaml
my_var: common


Test results searching for variable: my_var

# hiera my_var                                     <--without any specification it will be found in common.yaml
common

# hiera -d my_var ::fqdn=localhost.localdomain     <--with debug mode and specifying where to look
DEBUG: 2020-03-20 17:41:17 +0100: Hiera YAML backend starting
DEBUG: 2020-03-20 17:41:17 +0100: Looking up my_var in YAML backend
DEBUG: 2020-03-20 17:41:17 +0100: Looking for data source node/localhost.localdomain
DEBUG: 2020-03-20 17:41:17 +0100: Found my_var in node/localhost.localdomain
node

# hiera my_var osfamily=Debian                    <--specifying in osfamily which yaml file to check
debian

# hiera my_var osfamily=RedHat                    <--same as above, but in RedHat.yaml my_var is missing, found in common.yaml
common

# hiera tools::working_dir osfamily=Debian        <--checking a value os a variable in a class
/opt/debian

# hiera unknon_var                                <--if variable does not exit
nil

# hiera unknon_var 1111                           <--if variable does not exist give a default value to it
1111


================================