Downloading an Entire Web Site with wget

Ref: https://www.linuxjournal.com/content/downloading-entire-web-site-wget

If you ever need to download an entire Web site, perhaps for off-line viewing, wget can do the job—for example:

$ wget \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains website.org \
     --no-parent \
         www.website.org/tutorials/html/

This command downloads the Web site www.website.org/tutorials/html/.

The options are:

• --recursive: download the entire Web site.

• --domains website.org: don't follow links outside website.org.

• --no-parent: don't follow links outside the directory tutorials/html/.

• --page-requisites: get all the elements that compose the page (images, CSS and so on).

• --html-extension: save files with the .html extension.

• --convert-links: convert links so that they work locally, off-line.

• --restrict-file-names=windows: modify filenames so that they will work in Windows as well.

• --no-clobber: don't overwrite any existing files (used in case the download is interrupted and resumed).

Centos 8 not updating

 Ref: https://www.digitalocean.com/community/questions/cento-os-8-not-updating

Error: signature hdr data: BAD

I’ve reported this however for the time being what you can do is to update rpm alone and then do the dnf update on all packages;

dnf update rpm

After that is completed:

dnf update

 

CentOS through a VM - no URLs in mirrorlist

 

Ref: https://stackoverflow.com/questions/70926799/centos-through-a-vm-no-urls-in-mirrorlist

Check out this article: CentOS Linux EOL

The below commands helped me:

sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-Linux-*
sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-Linux-*

Doing this will make DNF work, but you will no longer receive any updates.

To upgrade to CentOS 8 stream:

sudo dnf install centos-release-stream -y
sudo dnf swap centos-{linux,stream}-repos -y
sudo dnf distro-sync -y

Optionally reboot if your kernel updated (not needed in containers).

 ref: https://gist.github.com/Bill-tran/5e2ab062a9028bf693c934146249e68c#file-how-to-install-openssl-1-1-1-on-centos-7-md

How To Install OpenSSL 1.1.1 on CentOS 7

This tutorial goes through how to install openssl 1.1.1 on CentOS 7, since the yum repo only installs up to openssl 1.0.

Requirements

Upgrade the system

yum -y update

Install required packages

yum install -y make gcc perl-core pcre-devel wget zlib-devel

Download the latest version of OpenSSL source code

wget https://ftp.openssl.org/source/openssl-1.1.1k.tar.gz

Configure, build and install OpenSSL

Uncompress the source file

tar -xzvf openssl-1.1.1k.tar.gz

Change to the OpenSSL directory

cd openssl-1.1.1k

Configure the package for compilation

./config --prefix=/usr --openssldir=/etc/ssl --libdir=lib no-shared zlib-dynamic

Compile package

make

Test compiled package

make test

Install compiled package

make install

Export library path

Create environment variable file

vim /etc/profile.d/openssl.sh

Add the following content

export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/lib64

Load the environment variable

source /etc/profile.d/openssl.sh

Verify the OpenSSL version

openssl version

How to change pids_limit value in OpenShift 4.x

 

Ref: https://computingforgeeks.com/change-pids-limit-value-in-openshift/

In OpenShift 4.x environment, each container running will be limited to the default maximum PID value of 1024. If there is a need to run applications with more than 1024 processes within one single container, the OpenShift Container Platform Cluster operator is required to adjust the default maximum PID value to a higher number.

pids_limit is the maximum number of processes allowed in a container, and it can be viewed by running the command below in an OpenShift node:

$ sudo crio-status config | grep pid
pids_limit = 1024

In OpenShift, it is NOT recommended to change the value directly by editing crio.conf file:

$ grep  pids_limit /etc/crio/crio.conf
pids_limit = 1024

It is recommended to follow the correct way that depends on the OCP version in use. I’m running OpenShift 4.4 and in this version, ContainerRuntimeConfig custom resource was introduced. You can check KCS Article 5133191.

Create ContainerRuntimeConfig custom resource for configuring cri-o pidsLimit

cat <<EOF > custom-pidslimit.yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: ContainerRuntimeConfig
metadata:
 name: custom-pidslimit
spec:
 machineConfigPoolSelector:
   matchLabels:
     custom-crio: custom-pidslimit
 containerRuntimeConfig:
   pidsLimit: 4096
EOF

You can update the configuration file before applying:

vim custom-pidslimit.yaml

Apply configuration:

$ oc create -f custom-pidslimit.yaml
containerruntimeconfig.machineconfiguration.openshift.io/custom-pidslimit created

Verify the resource has been created

$ oc get ctrcfg
NAME               AGE
custom-pidslimit   44s

Once the custom resource is created, we need to roll out the pidslimit changes to all the worker nodes in the cluster.

Let’s add custom-crio: custom-pidslimit under labels in the machineConfigPool config

$ oc edit machineconfigpool worker
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
  creationTimestamp: "2020-07-15T08:29:58Z"
  generation: 7
  labels:
    custom-crio: custom-pidslimit      #add this line

Check to ensure that a new 99-worker-XXX-containerruntime is created and that a new rendered worker is created:

$ oc get machineconfigs | grep containerruntime
99-worker-261cdd8d-c387-4f61-b1ce-b9ab2d025f09-containerruntime   601c2285f497bf7c73d84737b9977a0e697cb86a   2.2.0             93s

The changes should now be rolled out to each node in the worker pool via that new rendered-worker machine config.

You can verify by checking that the latest rendered-worker machine-config has been rolled out to the pools successfully:

$ oc get mcp
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-238bb9ffd94d526621cba8ee876c3ac8   True      False      False      5              5                   5                     0                      216d
worker   rendered-worker-6c236aa19af4d88fa0acdbc8f6ff53f3   False     True       True       10             0                   0                     7                      216d

Once a worker node is rebooted you can login and confirm the current setting:

$ oc debug node/<workernode>
sh-4.4# chroot /host
sh-4.4# grep pids_limit /etc/crio/crio.conf

I hope this short guide was helpful in changing the default value of pids_limit in your OpenShift 4.x cluster.

How to Upgrade Ubuntu 16.04 to Ubuntu 18.04

 

 

sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get dist-upgrade -y

sudo apt install update-manager-core -y

sudo do-release-upgrade

 

SQL test query or validation query

 

Ref: https://stackoverflow.com/questions/3668506/efficient-sql-test-query-or-validation-query-that-will-work-across-all-or-most

Many database connection pooling libraries provide the ability to test their SQL connections for idleness. For example, the JDBC pooling library c3p0 has a property called preferredTestQuery, which gets executed on the connection at configured intervals. Similarly, Apache Commons DBCP has validationQuery.

-- Access

SELECT 1 FROM (SELECT count(*) dual FROM MSysResources) AS dual

-- BigQuery, CockroachDB, Exasol, H2, Ignite, MariaDB, MySQL, PostgreSQL, 

-- Redshift, Snowflake, SQLite, SQL Server, Sybase ASE, Vertica

SELECT 1

-- MemSQL, Oracle

SELECT 1 FROM DUAL

-- CUBRID

SELECT 1 FROM db_root

-- Db2

SELECT 1 FROM SYSIBM.DUAL

-- Derby

SELECT 1 FROM SYSIBM.SYSDUMMY1

-- Firebird

SELECT 1 FROM RDB$DATABASE

-- HANA, Sybase SQL Anywhere

SELECT 1 FROM SYS.DUMMY

-- HSQLDB

SELECT 1 FROM (VALUES(1)) AS dual(dual)

-- Informix

SELECT 1 FROM (SELECT 1 AS dual FROM systables WHERE (tabid = 1)) AS dual

-- Ingres, Teradata

SELECT 1 FROM (SELECT 1 AS "dual") AS "dual"