Cloud, DevOps, Opensource 따라잡기

CNCF Landscape 가이드

freedream — Fri, 28 Apr 2023 13:16:54 +0900

Cloud Native application 및 기술을 조사했다면 CNCF에서 제공하는 Cloud Native 환경을 많이 접했을 것입니다.

CNCF Landscape 가이드는 이러한 많은 범주의 기술들을 기능 별로 깔금하게 구성하여 쉽게 탐색할 수 있게 해 줍니다.

1. Provisioning

프로비저닝은 클라우드 네이티브 환경의 첫 번째 계층입니다.

이 범주에는 클라우드 네이티브 앱이 구축되는 기반을 만들고 강화하는 데 사용되는 제품들이 포함됩니다.

• 인프라를 자동으로 구성, 생성 및 관리하고 컨테이너 이미지를 스캔, 서명 및 저장하는 도구를 찾을 수 있습니다.
• 정책 설정 및 시행, 내장된 인증 및 권한 부여, secret 배포 처리를 가능하게 하는 도구를 사용하여 보안으로 확장됩니다.

프로비저닝 계층은 인프라 프로비저닝에서 컨테이너 레지스트리, 보안에 이르기까지 모든 것을 처리하는 도구를 사용하여 클라우드 네이티브 플랫폼 및 애플리케이션의 기반을 구축하는 데 중점을 둡니다.

1) Automation & Configuration

자동화 및 구성 도구는 컴퓨팅 리소스(가상 머신, 네트워크, 방화벽 규칙, 로드 밸런서 등)의 생성 및 구성 속도를 높입니다.

이 범주의 제품은 프로비저닝 프로세스의 다른 부분을 처리하거나 모든 것을 종단 간 제어하는 기능을 제공합니다.

전통적으로 IT 프로세스는 일반적으로 3개월에서 6개월 사이의 길고 노동 집약적인 수동 릴리스 주기에 의존했습니다.

이 범주의 제품을 사용하면 자동화를 통해 컴퓨팅 환경을 구축할 수 있습니다.

Terraform과 같은 자동화된 도구는 수백 개의 방화벽 규칙으로 수십 개의 서버와 네트워크를 확장하는 데 필요한 노력을 줄여줍니다.

Puppet, Chef 및 Ansible과 같은 도구는 신규 서버와 애플리케이션을 프로그래밍 방식으로 프로비저닝 및 구성하여 개발자가 사용할 수 있도록 합니다.

Kubernetes 클러스터를 위한 컴퓨팅 환경, CPU, 메모리, 스토리지 및 네트워킹을 배치하는 과정의 일부로 하나 이상의 자동화 도구가 필요합니다.

범주	CNCF Projects
Infrastructure-as-Code (IaC) Automation Declarative Configuration	● Akri (sandbox) ● CDK for Kubernetes (CDK8s) (sandbox) ● Cloud Custodian (incubating) ● DevStream (sandbox) ● KubeDL (sandbox) ● KubeEdge (incubating) ● Metal (sandbox) ● OpenYurt (sandbox) ● SuperEdge (sandbox) ● Tinkerbell (sandbox)

2) Container Registry

클라우드 네이티브 애플리케이션은 패키징되어 컨테이너로 실행됩니다.

컨테이너 레지스트리는 이러한 앱을 실행하는 데 필요한 컨테이너 이미지를 저장하고 제공합니다.

모든 컨테이너 이미지를 한 곳에 중앙 집중식으로 저장하면 해당 앱에서 작업하는 모든 개발자가 쉽게 액세스할 수 있습니다.

기본적으로 레지스트리는 컨테이너 런타임이 이미지를 저장하고 검색할 수 있도록 하는 웹 API입니다.

이 범주의 제품들은 저장하는 이미지를 스캔, 서명 및 검사하기 위한 통합을 제공합니다.

Dragonfly와 Harbor는 CNCF 프로젝트이며 Harbor는 최근 최초의 OCI 준수 레지스트리라는 명성을 얻었습니다.

범주	CNCF Projects
Container OCI Image Registry	● Distribution (sandbox) ● Dragonfly (incubating) ● Harbor (graduated) ● zot (sandbox)

3) Security & Compliance

보안 및 규정 준수 도구는 플랫폼 및 애플리케이션 보안을 강화, 모니터링 및 적용하는 데 도움이 됩니다

컨테이너에서 Kubernetes 환경에 이르기까지 이러한 도구를 사용하면 규정 준수를 위한 정책을 설정하고, 기존 취약성에 대한 통찰력을 얻고, 잘못된 구성을 포착하고, 컨테이너 및 클러스터를 강화할 수 있습니다.

컨테이너를 안전하게 실행하려면 알려진 취약점이 있는지 컨테이너를 스캔하고 변조되지 않았는지 확인하기 위해 서명해야 합니다.

이 범주의 제품과 프로젝트는 클러스터를 강화하고 시스템이 비정상적으로 작동하는 시기를 감지하는 데 도움이 됩니다.

범주

CNCF Projects

Image scanning
Image signing
Policy enforcement
Audit
Certificate Management

● cert-manager (incubating)
● Confidential Containers (sandbox)
● ContainerSSH (sandbox)
● Curiefense (sandbox)
● Dex (sandbox)
● external-secrets (sandbox)
● Falco (incubating)
● Hexa (sandbox)
● in-toto (incubating)
● Keylime (sandbox)
● KubeArmor (sandbox)
● Kubescape (sandbox)
● Kubewarden (sandbox)
● Kyverno (incubating)
● Notary (incubating)
● Open Policy Agent (OPA) (graduated)
● Open Policy Containers (sandbox)
● OpenFGA (sandbox)
● Paralus (sandbox)
● Parsec (sandbox)
● The Update Framework (TUF) (graduated)
● Keycloak (incubating)

4) Key Management

애플리케이션과 운영이 새로운 클라우드 네이티브 환경에 적응함에 따라 보안 도구는 새로운 보안 요구 사항을 충족하도록 진화하고 있습니다.

이 범주의 제품 및 프로젝트는 암호 및 기타 Secret(API 키, 암호화 키 등과 같은 민감한 데이터)을 안전하게 저장하는 방법부터 마이크로 서비스 환경에서 암호 및 Secret을 안전하게 제거하는 방법에 이르기까지 모든 것을 다룹니다.

또한 Secret 및 key 또는 인증, 권한 부여와 관련된 서비스 또는 사양을 안전하게 배포하는 방법을 제공합니다.

범주	CNCF Projects
AuthN and AuthZ Identity Access Secrets	● Athenz (sandbox) ● SPIFFE (graduated) ● SPIRE (graduated) ● Teller (sandbox)

2. Runtime

이제 클라우드 네이티브 환경의 기반을 구축했으므로 하나의 인프라 계층을 위로 이동하여 런타임 계층으로 확대합니다.

런타임은 컨테이너가 클라우드 네이티브 환경에서 실행되는 데 필요한 모든 것을 포함합니다.

여기에는 컨테이너 런타임이라고 하는 컨테이너를 시작하는 데 사용되는 코드가 포함됩니다.

컨테이너 런타임은 컨테이너에서 영구 스토리지를 사용할 수 있도록 하고 컨테이너 환경의 네트워크 관리 기능을 제공합니다.

이 범주의 제품은 컨테이너를 시작 및 중지하고, 데이터를 저장하고, 서로 통신할 수 있도록 하는 데 사용됩니다.

1) Cloud Native Storage

Storage는 앱의 영구 데이터가 저장되는 곳으로 종종 persistent volume이라고 합니다.

애플리케이션이 안정적으로 작동하려면 스토리지에 쉽게 액세스할 수 있어야 합니다.

수동 프로비저닝과 자동 크기 조정은 호환되지 않으므로 클라우드의 탄력성을 활용하려면 스토리지를 자동으로 프로비저닝해야 합니다.

이 범주의 제품은 다음과 같은 기능을 제공합니다.

• 컨테이너용 클라우드 네이티브 스토리지 옵션 제공
• 컨테이너와 스토리지 공급자 간의 인터페이스를 표준화
• 백업 및 복원 작업을 통해 데이터 보호를 제공

클라우드 네이티브 스토리지는 컨테이너에 파일 및 블록 스토리지를 제공하기 위한 표준 API를 제공하는 컨테이너 스토리지 인터페이스(CSI)에 의해 크게 가능해졌습니다.

이 범주에는 CSI를 활용하여 컨테이너용 on-demand 스토리지를 제공하는 오픈 소스 및 벤더 제공 도구가 많이 있습니다.

Minio는 객체 스토리지를 위한 S3 호환 API를 제공하는 인기 있는 프로젝트입니다.

Velero는 Kubernetes 클러스터 자체와 애플리케이션에서 사용하는 영구 데이터의 백업 및 복원 프로세스를 단순화하는 데 도움이 됩니다.

범주	CNCF Projects
Persistent volume CSI Storage API Backup and restore	● Carina (sandbox) ● CubeFS (incubating) ● Curve (sandbox) ● K8up (sandbox) ● Longhorn (incubating) ● OpenEBS (sandbox) ● ORAS (sandbox) ● Piraeus Datastore (sandbox) ● Rook (graduated) ● Vineyard (sandbox)

2) Container Runtime

컨테이너 런타임은 컨테이너화된 애플리케이션을 실행하는 소프트웨어입니다.

런타임이 없으면 컨테이너화된 앱의 모양을 지정하는 저장 파일인 컨테이너 이미지만 갖게 됩니다.

런타임은 컨테이너 내에서 앱을 시작하고 필요한 리소스를 제공합니다.

이 범주의 제품은 다음과 같은 기능을 제공합니다.

• 컨테이너 이미지는 표준화되고 안전하며 격리된 방식으로 실행
• 모든 환경에서 표준화된 방식으로 앱을 시작하고 보안 경계를 설정
• 앱 간에 영향을 미치거나 받지 않도록 격리
• application에 CPU, 스토리지 및 메모리와 같은 리소스 제한 설정

범주	CNCF Projects
Container MicroVM	● containerd (graduated) ● CRI-O (incubating) ● Inclavare Containers (sandbox) ● Lima (sandbox) ● rkt (archived) ● WasmEdge Runtime (sandbox)

3) Cloud Native Network

컨테이너는 클라우드 네이티브 네트워크를 통해 서로 통신하고 인프라 계층과 통신합니다.

분산 응용 프로그램에는 다양한 목적으로 네트워크를 사용하는 여러 구성 요소가 있습니다.

이 범주의 도구는 앱이 통신할 수 있도록 특별히 기존 네트워크 위에 가상 네트워크를 만듭니다. 이를 overlay network라고 합니다.

클라우드 네이티브 네트워킹은 데이터 흐름을 제어, 검사 및 수정하기 위해 소프트웨어를 사용하기 때문에 컨테이너 간의 연결을 관리, 보호 및 격리하는 것이 훨씬 쉽습니다.

경우에 따라 방화벽 및 액세스 규칙과 같은 컨테이너 네트워크 및 네트워크 정책을 확장하여 앱이 컨테이너 네트워크 외부에서 실행되는 가상 머신 또는 서비스에 연결할 수 있도록 할 수 있습니다.

클라우드 네이티브 네트워킹의 프로그래밍 가능하고 종종 선언적인 특성이 이를 가능하게 합니다.

이 범주의 프로젝트 및 제품은 CNCF 프로젝트인 CNI(컨테이너 네트워크 인터페이스)를 사용하여 컨테이너화된 애플리케이션에 네트워킹 기능을 제공합니다.

Flannel과 같은 일부 도구는 다소 미니멀하여 컨테이너에 기본 연결을 제공합니다.

NSX-T와 같은 기타 제품은 모든 Kubernetes 네임스페이스에 대해 격리된 가상 네트워크를 생성하는 전체 소프트웨어 정의 네트워킹 계층을 제공합니다.

이 공간의 다양성과 혁신은 주로 CNI(위에서 언급한 스토리지 및 컨테이너 스토리지 인터페이스와 유사)에 의해 가능해졌습니다.

CNI는 네트워크 계층이 포드에 기능을 제공하는 방식을 표준화합니다.

Kubernetes 환경에 적합한 컨테이너 네트워크를 선택하는 것이 중요하며 선택할 수 있는 도구가 많이 있습니다.

Weave Net, Antrea, Calico 및 Flannel은 모두 효과적인 오픈 소스 네트워킹 계층을 제공합니다.

범주	CNCF Projects
SDN Network Overlay CNI	● Antrea (sandbox) ● Cilium (incubating) ● CNI-Genie (sandbox) ● Container Network Interface (CNI) (incubating) ● FabEdge (sandbox) ● Kube-OVN (sandbox) ● Network Service Mesh (sandbox) ● Submariner (sandbox)

3. Orchestration & Management

프로비저닝 및 런타임 레이어를 모두 다루었으므로 이제 오케스트레이션 및 관리에 대해 자세히 알아볼 수 있습니다.

여기에서 클라우드 네이티브 애플리케이션 실행 및 연결을 처리하는 도구를 찾을 수 있습니다.

이 섹션에서는 클라우드 네이티브 개발의 핵심 인에이블러 중 하나인 쿠버네티스 자체부터 앱 간 및 외부 통신을 담당하는 인프라 계층에 이르기까지 모든 것을 다룹니다.

1) Scheduling & Orchestration

오케스트레이션 및 스케줄링은 클러스터 전체에서 컨테이너를 실행하고 관리하는 것을 의미합니다.

클러스터는 네트워크를 통해 연결된 물리적 또는 가상 머신 그룹입니다.

클라우드 네이티브 아키텍처에서 애플리케이션은 각각 컨테이너에 배치되는 마이크로 서비스로 나뉩니다.

이러한 각 서비스에는 문제가 발생하면 리소스, 모니터링 및 수정이 필요합니다.

단일 서비스에 대해 이러한 모든 작업을 수동으로 수행하는 것이 가능할 수 있지만 각각 자체 컨테이너가 있는 여러 서비스를 처리할 때는 자동화된 프로세스가 필요합니다.

Kubernetes는 가장 일반적으로 사용되고 적극적으로 유지 관리되는 오케스트레이터입니다.

범주	CNCF Projects
Cluster Scheduler Orchestration	● Armada (sandbox) ● Clusternet (sandbox) ● Clusterpedia (sandbox) ● Crossplane (incubating) ● Fluid (sandbox) ● Karmada (sandbox) ● kube-rs (sandbox) ● Kubernetes (graduated) ● Kured (sandbox) ● Open Cluster Management (sandbox) ● Volcano (incubating) ● wasmCloud (sandbox)

2) Coordination & Service Discovery

클라우드 네이티브 아키텍처는 동적이고 유동적이므로 끊임없이 변화합니다.

컨테이너가 한 노드에서 충돌하면 이를 대체하기 위해 새 컨테이너가 다른 노드에서 가동됩니다. 또는 앱이 확장되면 복제본이 네트워크 전체에 분산됩니다.

특정 서비스가 있는 곳은 없습니다. 모든 것의 위치는 끊임없이 변화합니다.

이 범주의 도구는 필요할 때 서비스가 서로를 찾을 수 있도록 네트워크 내의 서비스를 추적합니다.

서비스 검색 도구는 개별 서비스를 찾고 잠재적으로 식별할 수 있는 공통 위치를 제공하여 이 문제를 해결합니다.

이 범주에는 기본적으로 두 가지 유형의 도구가 있습니다.

• Service discovery engine: 모든 서비스에 대한 정보와 이를 찾는 방법을 저장하는 데이터베이스와 같은 도구
• Name resolution tool: 서비스 위치 요청을 수신하고 네트워크 주소 정보를 반환하는 도구 (예: CoreDNS)

분산 시스템이 점점 더 보편화됨에 따라 기존 DNS 프로세스와 로드 밸런서는 변화하는 엔드포인트 정보를 따라잡지 못하는 경우가 많았습니다.

이러한 단점을 보완하기 위해 서비스 검색 도구는 개별 응용 프로그램 인스턴스를 신속하게 등록 및 등록 취소합니다.

CoreDNS 및 etcd와 같은 일부 옵션은 CNCF 프로젝트이며 Kubernetes에 내장되어 있습니다.

범주	CNCF Projects
DNS Service Discovery	● CoreDNS (graduated) ● etcd (graduated) ● k8gb (sandbox)

3) Remote Procedure Call

RPC(원격 프로시저 호출)는 응용 프로그램이 서로 통신할 수 있도록 하는 특정 기술입니다.

앱 커뮤니케이션을 구성하는 한 가지 방법입니다.

최신 앱은 협업을 위해 통신해야 하는 수많은 개별 서비스로 구성됩니다.

RPC는 응용 프로그램 간의 통신을 처리하기 위한 옵션 중 하나입니다.

RPC는 서비스 간의 통신을 처리하는 긴밀하게 결합되고 독단적인 방식을 제공합니다.

대역폭 효율적인 통신이 가능하고 많은 프로그래밍 언어로 RPC 인터페이스 구현이 가능합니다.

RPC는 코딩 연결을 더 쉽게 만들고 네트워크 계층을 매우 효율적으로 사용하고 서비스 간에 잘 구성된 통신을 허용합니다.

gRPC는 특히 널리 사용되는 RPC 구현이며 CNCF에서 채택했습니다.

범주	CNCF Projects
gRPC	● gRPC (incubating)

4) Service Proxy

서비스 프록시는 지정된 서비스에서 들어오고 나가는 트래픽을 가로채고 일부 논리를 적용한 다음 해당 트래픽을 다른 서비스로 전달하는 도구입니다.

기본적으로 네트워크 트래픽에 대한 정보를 수집하고 규칙을 적용할 수 있는 "중개자" 역할을 합니다.

이는 트래픽을 개별 애플리케이션으로 전달하는 로드 밸런서 역할을 하는 것처럼 간단할 수도 있고, 모든 네트워크 연결을 처리하는 컨테이너화된 개별 애플리케이션과 나란히 실행되는 프록시의 상호 연결된 메시처럼 복잡할 수도 있습니다.

서비스 프록시는 데이터 수집 및 네트워크 트래픽 관리 기능을 "외부화"합니다.

서비스 프록시 위치는 앱 내에 있지 않고 플랫폼 계층에 포함됩니다.

이는 개발자가 애플리케이션 코드를 작성하는 데 전적으로 집중할 수 있도록 하여 트래픽을 처리하는 보편적인 작업을 플랫폼 팀이 관리할 수 있도록 해줍니다.

하나의 공통 위치에서 Routing 또는 TLS termination와 같이 필요한 서비스 기능의 배포 및 관리를 중앙 집중화하면 서비스 간의 통신이 보다 안정적이고 안전하며 성능이 향상될 수 있습니다.

프록시는 중요한 데이터를 수집하고 서비스 간에 트래픽을 고르게 분산하거나 서비스 중단 시 재 라우팅하는 등의 라우팅 관리 기능을 하고, 연결을 암호화하고 콘텐츠를 캐시합니다.

범주	CNCF Projects
Service Proxy Ingress	● BFE (sandbox) ● Contour (incubating) ● Envoy (graduated) ● OpenELB (sandbox)

5) API Gateway

API 게이트웨이를 사용하면 조직에서 애플리케이션 간의 요청 수를 승인하거나 제한하는 것과 같은 주요 기능을 중앙에서 관리되는 위치로 이동할 수 있습니다.

또한 API 소비자에 대한 공통 인터페이스 기능을 합니다.

API 게이트웨이를 사용하면 개발자가 더 적은 사용자 정의 코드를 작성하고 유지할 수 있습니다.

API 게이트웨이는 사용자와 애플리케이션 사이에 있습니다.

사용자의 메시지(요청)를 받아 적절한 서비스로 전달하는 중개자 역할을 합니다.

요청을 전달하기 전에 사용자가 하려는 작업을 수행할 수 있는지 여부를 평가하고 요청한 사람과 요청한 횟수에 대한 세부 정보를 기록합니다.

API 게이트웨이는 일련의 다운스트림 애플리케이션에 대한 공통 진입점 역할을 하는 동시에 팀이 비즈니스 로직을 추가하여 승인, 속도 제한 및 지불 거절을 처리할 수 있는 장소를 제공합니다.

이를 통해 애플리케이션 개발자는 고객의 다운스트림 API에 대한 변경 사항을 추상화하고 신규 고객 온보딩과 같은 작업을 게이트웨이로 오프로드할 수 있습니다.

범주	CNCF Projects
API Gateway	● Emissary-Ingress (incubating)

6) Service Mesh

서비스 메시는 서비스 간의 트래픽을 관리합니다.

이를 통해 플랫폼 팀은 코드를 변경할 필요 없이 클러스터 내에서 실행되는 모든 서비스에 걸쳐 안정성, 관찰 가능성 및 보안 기능을 균일하게 추가할 수 있습니다.

Kubernetes와 함께 서비스 메시는 클라우드 네이티브 스택의 가장 중요한 인프라 구성 요소 중 일부가 되었습니다.

서비스 메시는 앱 코드를 건드리지 않고 플랫폼 계층의 모든 서비스에 걸쳐 안정성, 관찰 가능성 및 보안 기능을 균일하게 추가합니다.

모든 프로그래밍 언어와 호환되므로 개발 팀이 비즈니스 로직 작성에 집중할 수 있습니다.

명령 및 제어 신호를 서비스 프록시 네트워크에 제공하여 서비스 간 통신을 관리하는 인프라 계층입니다.

범주	CNCF Projects
Service mesh Sidecar Data plane Control plane	● Aeraki Mesh (sandbox) ● Istio (incubating) ● Kuma (sandbox) ● Linkerd (graduated) ● Merbridge (sandbox) ● Meshery (sandbox) ● Open Service Mesh (sandbox) ● Service Mesh Interface (SMI) (sandbox) ● Service Mesh Performance (sandbox)

4. App Definition and Development

CNCF 환경의 마지막 계층인 애플리케이션 정의 및 개발 계층에 대해 설명합니다.

database, streaming 및 messaging, 애플리케이션 정의, CICD(continuous integration and continuous delivery)를 다룹니다.

지금까지 다루었던 내용은 안정적이고 안전한 환경을 구축하고 필요한 모든 종속성을 제공하는 것과 관련이 있습니다.

이제 CNCF 클라우드 네이티브 환경의 최상위 계층에 도달했습니다.

응용 프로그램 정의 및 개발 계층은 엔지니어가 응용 프로그램을 구축할 수 있도록 하는 도구에 중점을 둡니다.

1) Database

데이터베이스는 다른 앱이 데이터를 효율적으로 저장하고 검색할 수 있는 애플리케이션입니다. 데이터베이스를 사용하면 데이터를 저장하고 권한이 있는 사용자만 데이터에 액세스할 수 있으며 사용자가 특수 요청을 통해 데이터를 검색할 수 있습니다.

Kubernetes의 부상과 stateful application 지원 기능으로 차세대 데이터베이스가 컨테이너화를 활용하는 것을 볼 수 있습니다.

클라우드 네이티브 데이터베이스는 Kubernetes의 확장성 및 가용성 이점을 데이터베이스에 제공하는 것을 목표로 합니다.

클라우드 네이티브 데이터베이스로 YugabyteDB 및 Couchbase 등을 들 수 있습니다만,

MySQL과 Postgres와 같은 보다 전통적인 데이터베이스도 Kubernetes 클러스터 환경에서 성공적이고도 효율적으로 실행되는 것을 볼 수 있습니다.

범주	CNCF Projects
SQL DB Persistence	● SchemaHero (sandbox) ● TiKV (graduated) ● Vitess (graduated)

2) Streaming & Messaging

스트리밍 및 메시징 도구는 시스템 간에 메시지를 전송하여 서비스 간 통신을 가능하게 합니다.

스트리밍 또는 메시징 플랫폼은 시스템 내에서 발생하는 모든 이벤트를 게시하고 읽을 수 있는 중앙 위치를 제공하므로 응용 프로그램이 서로에 대해 전혀 알지 못하는 상태에서 함께 작동할 수 있습니다.

NATS 및 Cloudevents 프로젝트는 CNCF 프로젝트를 인큐베이팅하고 있습니다.

NATS는 성숙한 메시징 시스템을 제공하고 Cloudevents는 시스템 간의 메시지 형식을 표준화하려는 노력입니다.

Strimzi, Pravega 및 Tremor는 각각 스트리밍 및 메시징과 관련된 고유한 사용 사례에 맞게 조정되는 샌드박스 프로젝트입니다.

범주	CNCF Projects
Choreography Streaming MQ Message bus	● CloudEvents (incubating) ● NATS (incubating) ● Pravega (sandbox) ● Strimzi (sandbox) ● Tremor (sandbox)

3) Application Definition & Image Build

애플리케이션 정의 및 이미지 빌드는 두 개의 주요 하위 그룹으로 나눌 수 있는 광범위한 범주입니다.

첫째, 컨테이너 및/또는 Kubernetes에 애플리케이션 코드를 빌드하는 데 도움이 되는 개발자 중심 도구입니다.

둘째, 표준화된 방식으로 앱을 배포하는 운영 중심 도구입니다.

개발 환경의 속도를 높이거나 단순화하거나 third-party앱을 배포하는 표준화된 방법을 제공하거나 새로운 Kubernetes 확장 작성 프로세스를 단순화하려는 경우, 이 범주는 Kubernetes 개발자 및 운영자 경험을 최적화하는 여러 프로젝트 및 제품에 대한 포괄적인 역할을 합니다.

Kubernetes 및 보다 일반적으로 컨테이너화된 환경은 매우 유연하고 강력합니다.

이러한 유연성은 주로 여러 구성 옵션의 형태와 다양한 사용 사례에 대한 여러 요구 사항의 형태로 복잡성도 수반합니다.

• 개발자는 코드를 컨테이너화할 때 재현 가능한 이미지를 생성할 수 있는 기능이 필요합니다.
• 운영자는 컨테이너 환경에 앱을 배포하는 표준화된 방법이 필요하며,
• 플랫폼 팀은 사내 및 타사 애플리케이션 모두에 대해 이미지 생성 및 애플리케이션 배포를 단순화하는 도구를 제공해야 합니다.

이 범주의 도구는 이러한 개발자 또는 운영자 문제 중 일부를 해결하는 것을 목표로 합니다.

개발자 측에는 Kubernetes를 확장하여 애플리케이션을 빌드, 배포 및 연결하는 프로세스를 간소화하는 도구가 있습니다.

Helm을 사용하면 Kubernetes 사용자가 많은 인기 있는 타사 앱을 배포하고 사용자 지정할 수 있으며 Artifact Hub(CNCF 샌드박스 프로젝트)와 같은 다른 프로젝트에서 채택되었습니다.

Bitnami와 같은 회사도 엄선된 앱 카탈로그를 제공합니다.

Operator Framework는 Operator 구축 및 배포 프로세스를 단순화하는 것을 목표로 하는 인큐베이팅 프로젝트입니다.

범주

CNCF Projects

Package Management
Charts
Operators

● Artifact Hub (sandbox)
● Backstage (incubating)
● Buildpacks (incubating)
● Carvel (sandbox)
● Devfile (sandbox)
● DevSpace (sandbox)
● Helm (graduated)
● ko (sandbox)
● Konveyor (sandbox)
● Krator (sandbox)
● KubeVela (incubating)
● KubeVirt (incubating)
● KUDO (sandbox)
● Nocalhost (sandbox)
● Operator Framework (incubating)
● Porter (sandbox)
● sealer (sandbox)
● Serverless Workflow (sandbox)
● Telepresence (sandbox)

4) Continuous Integration & Delivery

CI(Continuous integration) 및 CD(continuous delivery) 도구를 사용하면 내장된 품질 보증을 통해 빠르고 효율적인 개발이 가능합니다.

CI는 코드를 즉시 빌드하고 테스트하여 코드 변경을 자동화하여 배포 가능한 아티팩트를 생성하도록 합니다.

CD는 한 단계 더 나아가 배포 단계를 통해 artifact를 push합니다.

성숙한 CI/CD 시스템은 소스 코드의 변경 사항을 감시하고 코드를 자동으로 빌드 및 테스트한 다음 프로세스를 계속할지 또는 실패할지 결정하기 위해 다양한 테스트 또는 유효성 검사를 통과해야 하는 개발에서 프로덕션으로 이동하기 시작합니다.

이 범주의 도구는 이러한 접근 방식을 가능하게 합니다.

시장에서 가장 많은 CI 도구인 Jenkins와 같은 일부 기존 도구는 Kubernetes 생태계에 더 잘 맞도록 조정되어 있습니다.

Flux 및 Argo는 OpenGitOps 프로젝트가 벤더 중립적 표준으로 정의하기 위해 노력하고 있는 GitOps라는 새로운 지속적 제공 방법을 개척했습니다.

범주	CNCF Projects
CI/CD Continuous integration Continuous delivery Continuous deployment Blue/green Canary deploy	● Argo (graduated) ● Brigade (archived) ● Flux (graduated) ● Keptn (incubating) ● OpenFeature (sandbox) ● OpenGitOps (sandbox) ● OpenKruise (incubating) ● werf (sandbox)

5. Observability and Analysis

이제 CNCF 환경의 계층을 통해 작업했으므로 관측 가능성 및 분석으로 시작하는 열에 초점을 맞출 것입니다.

Observability은 외부 출력에서 시스템을 이해할 수 있는 정도를 설명하는 시스템 특성입니다.

CPU 시간, 메모리, 디스크 공간, 대기 시간, 오류 등으로 측정되는 컴퓨터 시스템은 어느 정도 관찰 가능할 수 있습니다.

Analysis는 관찰 가능한 데이터를 보고 이해하는 활동입니다.

서비스 중단이 발생하지 않도록 하려면 애플리케이션의 모든 측면을 관찰하고 분석하여 모든 이상 현상을 즉시 감지하고 수정해야 합니다.

이 범주의 도구는 로깅, 모니터링, 추적 및 카오스 엔지니어링으로 분류됩니다.

1) Monitoring

모니터링은 동작에 대한 이해도를 높이기 위해 로그와 메트릭을 수집, 집계 및 분석하기 위해 앱을 계측하는 것을 의미합니다.

로그는 특정 이벤트를 설명하지만 메트릭은 지정된 시점의 시스템 측정치입니다.

두 가지가 서로 다르지만 둘 다 시스템 상태를 전체적으로 파악하는 데 필요합니다.

모니터링에는 개별 노드의 디스크 공간, CPU 사용량 및 메모리 소비를 관찰하는 것부터 상세한 가상 트랜잭션을 수행하여 시스템 또는 애플리케이션이 적시에 올바르게 응답하는지 확인하는 것까지 모든 것이 포함됩니다.

시스템 및 응용 프로그램을 모니터링하는 방법에는 여러 가지가 있습니다.

범주	CNCF Projects
Monitoring Time series Alerting Metrics	● Cortex (incubating) ● Fonio (sandbox) ● Kuberhealthy (sandbox) ● OpenMetrics (incubating) ● Pixie (sandbox) ● Prometheus (graduated) ● Skooner (sandbox) ● Thanos (incubating) ● Trickster (sandbox)

2) Logging

애플리케이션은 주어진 시간에 수행 중인 작업을 설명하는 꾸준한 로그 메시지 스트림을 내보냅니다.

이러한 로그 메시지는 실패하거나 성공한 작업, 감사 정보 또는 상태 이벤트와 같이 시스템에서 발생하는 다양한 이벤트를 캡처합니다.

로깅 도구는 이러한 메시지를 수집, 저장 및 분석하여 오류 보고서 및 관련 데이터를 추적합니다.

메트릭 및 추적과 함께 로깅은 관찰 가능성의 기둥 중 하나입니다.

클라우드 네이티브 환경에서 Fluentd와 같은 로그 수집 도구는 애플리케이션 컨테이너와 함께 실행되며 애플리케이션에서 직접 메시지를 수집합니다.

그런 다음 메시지는 집계 및 분석을 위해 중앙 로그 저장소로 전달됩니다.

범주	CNCF Projects
Logging	● Fluentd (graduated)

3) Tracing

Logging의 전문적인 사용인 Tracing을 통해 분산 시스템을 통해 이동할 때 요청 경로를 추적할 수 있습니다.

Tracing은 응용 프로그램에서 보낸 메시지에 고유 식별자를 추가하여 이 문제를 해결합니다.

이 고유 식별자를 사용하면 개별 트랜잭션이 시스템을 통해 이동할 때 추적할 수 있습니다.

이 정보를 사용하여 애플리케이션의 상태를 확인하고 문제가 있는 마이크로 서비스 또는 활동을 디버그할 수 있습니다.

Tracing 데이터를 내보내려면 응용 프로그램 코드를 수정해야 하며 모든 범위는 응용 프로그램의 데이터 경로에서 인프라 구성 요소(예: 서비스 메시 및 해당 프록시)에 의해 전파되어야 합니다.

Jaeger 및 Open Tracing은 이 분야의 CNCF 프로젝트입니다.

범주	CNCF Projects
Span Tracing	● Jaeger (graduated) ● OpenTelemetry (incubating) ● OpenTracing (archived)

4) Chaos Engineering

카오스 엔지니어링은 탄력성을 테스트하고 응용 프로그램 및 엔지니어링 팀이 예기치 않은 이벤트를 견딜 수 있도록 시스템에 의도적으로 결함을 도입하는 관행을 말합니다.

카오스 엔지니어링 도구는 애플리케이션의 특정 인스턴스에 대해 결함을 도입하고 특정 실험을 실행하는 제어된 방법을 제공합니다.

범주	CNCF Projects
Chaos Engineering	● Chaos Mesh (incubating) ● Chaosblade (sandbox) ● Litmus (incubating)

6. Platform

플랫폼은 오케스트레이션 도구, 컨테이너 런타임, 서비스 검색, 네트워킹, API 게이트웨이 등의

서로 다른 계층의 서로 다른 도구를 함께 묶어 제공합니다.

소규모 엔지니어링 팀이 있는 조직의 경우 플랫폼이 클라우드 네이티브 접근 방식을 채택하는 유일한 방법입니다.

모든 플랫폼이 클라우드 네이티브 스택의 핵심인 Kubernetes를 중심으로 구성되어 가고 있습니다.

1) Certified Kubernetes - Distribution

배포판은 공급업체가 core Kubernetes를 가져와 재 배포를 위해 패키징하는 것입니다.

일반적으로 여기에는 Kubernetes 소프트웨어를 찾아 검증하고 클러스터 설치 및 업그레이드를 처리하는 메커니즘을 제공하는 것이 수반됩니다.

많은 Kubernetes 배포에는 공급업체 별 독점적이거나 오픈 소스 애플리케이션이 포함됩니다.

범주	CNCF Projects
	● k3s (sandbox)

2) Certified Kubernetes - Hosted

호스팅된 Kubernetes는 AWS, Digital Ocean, Azure 및 Google과 같은 인프라 공급자가 제공하는 서비스로 고객이 필요에 따라 Kubernetes 클러스터를 가동할 수 있습니다.

클라우드 공급자는 일반적으로 컨트롤 플레인이라고 하는 Kubernetes 클러스터의 일부를 관리할 책임이 있습니다.

배포판과 비슷하지만 인프라에서 클라우드 공급자가 관리합니다.

공급자가 모든 관리 세부 사항을 처리하므로 호스팅된 Kubernetes는 클라우드 네이티브를 시작하는 가장 쉬운 방법입니다.

모든 사용자는 앱을 개발하고 호스팅된 Kubernetes 서비스에 배포하기만 하면 됩니다.

오퍼링은 클라우드 공급자에 바인딩되며 Kubernetes 사용자는 컨트롤 플레인에 액세스할 수 없습니다.

3) Certified Kubernetes - Installer

Kubernetes 설치 프로그램은 컴퓨터에 Kubernetes를 설치하는 데 도움이 됩니다.

Kubernetes 설치 및 구성 프로세스를 자동화하고 업그레이드를 지원할 수도 있습니다.

Kubernetes 설치 프로그램은 종종 Kubernetes 배포 또는 호스팅된 Kubernetes 제품과 결합되거나 사용됩니다.

Kubernetes 설치 프로그램은 Kubernetes 설치 프로세스를 용이하게 합니다.

배포판과 마찬가지로 소스 코드 및 버전에 대해 검증된 소스를 제공합니다.

또한 공급업체 별 독단적인 Kubernetes 환경 구성과 함께 제공되는 경우가 많습니다.

KIND (Docker의 Kubernetes) 와 같은 Kubernetes 설치 프로그램을 사용하면 단일 명령으로 Kubernetes 클러스터를 가져올 수 있습니다.

4) PaaS/Container Service

PaaS(Platform-as-a-Service)는 사용자가 기본 컴퓨팅 리소스의 세부 정보에 신경 쓰지 않고 애플리케이션을 실행할 수 있는 환경입니다.

이 범주의 PaaS 및 컨테이너 서비스는 개발자용 PaaS를 호스팅하거나 개발자가 사용할 수 있는 서비스를 호스팅하는 메커니즘입니다.

※ 참고 URL

https://landscape.cncf.io/guide

Install ELK Stack on CentOS 7

freedream — Wed, 19 Aug 2020 17:52:08 +0900

Install ELK Stack on CentOS 7

How can I install ELK Stack on CentOS 7 / Fedora ? “ELK” is the acronym for Elasticsearch, Logstash, and Kibana. A short description of these tools is covered in the next block.

Elasticsearch: This is an open source, distributed, RESTful, JSON-based search engine. It is scalable, easy to use, and flexible
Logstash : This is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a “stash” like Elasticsearch.
Kibana lets users visualize data with charts and graphs in Elasticsearch.

For RHEL 8, refer to:

How to Install ELK Stack on RHEL / CentOS 8

Please follow our steps below to install and configure ELK stack tools on CentOS 7 / Fedora 31/30/29 Linux.

Step 1: Install Java

As Elasticsearch depends on Java, you need to install Java on your CentOS 7 / Fedora system.

sudo yum -y install java-openjdk-devel java-openjdk

Step 2: Add ELK repository

Once you have Java installed, add ELK stack repository which provides ELK stack packages.

For Elasticsearch 7.x

cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo

[elasticsearch-7.x]

name=Elasticsearch repository for 7.x packages

baseurl=https://artifacts.elastic.co/packages/7.x/yum

gpgcheck=1

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

EOF

For Elasticsearch 6.x

cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo

[elasticsearch-6.x]

name=Elasticsearch repository for 6.x packages

baseurl=https://artifacts.elastic.co/packages/6.x/yum

gpgcheck=1

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

EOF

After adding the repo, import GPG key:

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Clear and update your YUM package index.

sudo yum clean all

sudo yum makecache

Step 3: Install and Configure Elasticsearch

Elasticsearch repository is ready for use. You can install Elasticsearch using the command below:

sudo yum -y install elasticsearch

Confirm package installation.

$ rpm -qi elasticsearch

Name : elasticsearch

Epoch : 0

Version : 7.0.1

Release : 1

Architecture: x86_64

Install Date: Mon 06 May 2019 09:59:57 PM EAT

Group : Application/Internet

Size : 571521653

License : Elastic License

Signature : RSA/SHA512, Mon 29 Apr 2019 05:14:11 PM EAT, Key ID d27d666cd88e42b4

Source RPM : elasticsearch-7.0.1-1-src.rpm

Build Date : Mon 29 Apr 2019 04:06:59 PM EAT

Build Host : packer-virtualbox-iso-1553723689

Relocations : /usr

Packager : Elasticsearch

Vendor : Elasticsearch

URL : https://www.elastic.co/

Summary : Elasticsearch is a distributed RESTful search engine built for the cloud. Reference documentation can be found at https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html and the 'Elasticsearch: The Definitive Guide' book can be found at https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html

Description :

Elasticsearch subproject :distribution:packages

You can set JVM options like memory limits by editing the file: /etc/elasticsearch/jvm.options

Example below sets initial/maximum size of total heap space

-Xms1g

-Xmx1g

If your system has less memory, you can configure it to use small megabytes of ram.

-Xms256m

-Xmx512m

Start and enable elasticsearch service on boot:

$ sudo systemctl enable --now elasticsearch.service

Synchronizing state of elasticsearch.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.

Executing: /usr/lib/systemd/systemd-sysv-install enable elasticsearch

Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /usr/lib/systemd/system/elasticsearch.service.

Test to verify that it is working:

$ curl http://127.0.0.1:9200

{

"name" : "bBzN5Kg",

"cluster_name" : "elasticsearch",

"cluster_uuid" : "LKyqXXSvRvCpX9QAwKlP2Q",

"version" : {

"number" : "6.5.4",

"build_flavor" : "default",

"build_type" : "rpm",

"build_hash" : "d2ef93d",

"build_date" : "2018-12-17T21:17:40.758843Z",

"build_snapshot" : false,

"lucene_version" : "7.5.0",

"minimum_wire_compatibility_version" : "5.6.0",

"minimum_index_compatibility_version" : "5.0.0"

"tagline" : "You Know, for Search"

}

Create a test index:

$ curl -X PUT "http://127.0.0.1:9200/mytest_index"

{"acknowledged":true,"shards_acknowledged":true,"index":"mytest_index"}

Step 4: Install and Configure Kibana

Download and install Kibana from the added Elasticsearch repository.

sudo yum -y install kibana

After a successful installation, configure Kibana:

$ sudo vim /etc/kibana/kibana.yml

server.host: "0.0.0.0"

server.name: "kibana.example.com"

elasticsearch.hosts: "http://localhost:9200"

Change other settings as desired then start kibana service:

sudo systemctl enable --now kibana

Access http://ip-address:5601 to open Kibana Dashboard:

If you have an active firewall service, allow TCP port 5601

sudo firewall-cmd --add-port=5601/tcp --permanent

sudo firewall-cmd --reload

Step 5: Install and Configure Logstash

The last installation is for Logstash. It will act as a centralized logs server for your client systems which runs an agent like filebeat.

sudo yum -y install logstash

Logstash custom configurations can be placed under the /etc/logstash/conf.d/directory.

Check Logstash Configuration manual for more details.

Step 6: Install other ELK tools – Bonus

Other ELK tools that can be installed include:

Filebeat: Lightweight Shipper for Logs. It helps you keep the simple things simple by offering a lightweight way to forward and centralize logs and files
Metricbeat: Collect metrics from your systems and services. From CPU to memory, Redis to NGINX, and much more, Metricbeat is a lightweight way to send system and service statistics.
Packetbeat: Lightweight Shipper for Network Data
Heartbeat: Lightweight Shipper for Uptime Monitoring. It helps you monitor services for their availability with active probing
Auditbeat: Lightweight shipper that helps you audit the activities of users and processes on your systems

These tools can be installed with yum package manager using their respective names. The example below will install all ELK addon tools.

sudo yum install filebeat auditbeat metricbeat packetbeat heartbeat-elastic

Refer to official ELK stack documentation and Resources and Training for each tool configuration and further reading.

https://computingforgeeks.com/how-to-install-elk-stack-on-centos-fedora/

Install Python 3 on CentOS 7

freedream — Wed, 19 Aug 2020 17:49:14 +0900

How to Install Python 3 on CentOS 7

the pre-installed version of Python found on CentOS 7 is a much older version. In order to have the latest version of Python, the user will have to install it manually.

In this tutorial, we will show you two methods of installing Python version 3.6.8 on your CentOS 7 machine. The reason there are two methods is that Python 3.6.8 does not come by default in the built-in Yum repositories. So, we can either compile Python on our machine from the source code or add a repository that has a pre-compiled version of Python for CentOS 7.

Method 1: Install Python 3 on CentOS 7 From a Repository

This is the easier method of the two for installing Python on your machine. Here, we simply add a repository that has the pre-compiled version ready for us to install. In this case, we are adding the Inline with Upstream Stable repository, a community project whose goal is to bring new versions of software to RHEL-based systems.

Step 1: Open a Terminal and add the repository to your Yum install.

sudo yum install -y https://repo.ius.io/ius-release-el7.rpm

Step 2: Update Yum to finish adding the repository.

sudo yum update

Step 3: Download and install Python.

This will not only install Python – but it will also install pip to help you with installing add-ons.

sudo yum install -y python36u python36u-libs python36u-devel python36u-pip

Once these commands are executed, simply check if the correct version of Python has been installed by executing the following command:

python3.6 -V

You have now finished installing Python 3.6.8 on your CentOS 7 machine, as well as installing a native Python package management tool called pip.

sudo easy_install-3.6 pip

pip --version

Add python alias

which python3.6

/usr/bin/python3.6

sudo vi ~/.bashrc

alias python="/usr/bin/python3.6"

source ~/.bashrc

python --version

Python 3.6.8

Method 2: Compiling Python 3.7.9 on CentOS 7 From Source Code

This is the more complicated method of the two, and will take more time – however, this method gives you more control over what gets installed and what doesn’t. It can also be more secure at times, depending on where the software package is downloaded from.

Important: Keep in mind that your Yum package manager will not know that you have installed Python 3.7.9 (or any other software) if you install software by manually compiling the source code. This means that no updates will be available for your manually installed software.

Step 1: Install the development tools needed for compilation.

First, we will need the tools in order to be able to compile and install programs from their source code. To do this, we will install the group “Development Tools” through Yum itself:

sudo yum groupinstall -y "Development Tools"

Once this is done, move on to step 2.

Step 2: Download the Python source files.

First, we need to create a directory in which our install will take place. Make a directory with a name of your choosing, then enter the directory. Once you are in your new directory, enter the following command to download the compressed Python source file.

wget https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tar.xz

Once the file is finished downloading, uncompress the file by using tar, then enter into the new directory that was just created:

tar -xJf Python-3.7.9.tar.xz

cd Python-3.7.9

Step 3: Run the configuration script.

Use the following command to have the installation software check your system before actually starting the installation process.

./configure

This command ensures that the install will work, along with creating a special ‘makefile’ that is unique to your system. This makefile is what you will use to install Python onto your system.

Step 4: Install Python.

Now we can finally execute the makefile. Run the following command to install Python onto your system. Note: This will take a few minutes. The speed of the compilation and install will depend on the speed of your processor.

First, we run the ‘make’ command which compiles the program.

make

Then, once that is finished, we can run the installation command.

make install

Once this command finishes, you will have installed Python successfully, along with pip and setuptools. From here, creating a virtual environment is easy, and coding and executing the latest Python code is now possible.

Of course, you don’t have to install Python 3.7.9 on CentOS 7, if you use one of our high-speed Python VPS hosting plans – in which case you can simply ask our expert Linux admins to install Python 3.7.9 for you. They are available 24×7 and will take care of your request immediately.

https://www.rosehosting.com/blog/how-to-install-python-3-6-4-on-centos-7/

Kubernetes Monitoring - Prometheus

freedream — Wed, 19 Aug 2020 17:47:28 +0900

Kubernetes Monitoring - Prometheus 실습

Overview

이번 포스팅에서는 쿠버네티스 클러스터의 메트릭들을 프로메테우스로 수집하고 web UI를 통해 시각화 시키는 작업을 해보겠습니다.

참고 링크 : 쿠버네티스 시작하기(11) - Prometheus & Node-Exporter & AlertManager 연동

Prerequisites

먼저 쿠버네티스 클러스터를 생성해주세요.

참고링크 : 호롤리한하루/Install Kubernetes on CentOS/RHEL

본 실습에서 사용한 spec :

OS : CentOS v7.6

Arch : x86

Kubernetes : v1.16.2

Master : 4cpu, ram16G (1개)

Node : 4cpu, ram16G (2개)

Step

1. Prometheus 배포

namespace

제일먼저 해줘야 할 것은 프로메테우스관련 오브젝트들이 사용할 namespace를 생성해 주는 것입니다.

$ kubectl create ns monitoring

Cluster Role

다음으로는 프로메테우스 컨테이너가 쿠버네티스 api에 접근할 수 있는 권한을 부여해주기 위해 ClusterRole을 설정해주고 ClusterRoleBinding을 해줍니다.

생성된 ClusterRole은 monitoring namespace의 기본 서비스어카운트와 연동되어 권한을 부여해줍니다.

# prometheus-cluster-role.yaml

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: ClusterRole

metadata:

namespace: monitoring

rules:

- apiGroups: [""]

resources:

- nodes

- nodes/proxy

- services

- endpoints

- pods

verbs: ["get", "list", "watch"]

- apiGroups:

- extensions

resources:

- ingresses

verbs: ["get", "list", "watch"]

- nonResourceURLs: ["/metrics"]

verbs: ["get"]

---

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: ClusterRoleBinding

metadata:

roleRef:

apiGroup: rbac.authorization.k8s.io

kind: ClusterRole

subjects:

- kind: ServiceAccount

namespace: monitoring

Configmap

프로메테우스가 기동되려면 환경설정파일이 필요한데 해당 환경설정 파일을 정의해주는 부분입니다.

data 밑에 prometheus.rules와 prometheus.yml를 각각 정의하게 되어있습니다.

prometheus.rules : 수집한 지표에 대한 알람조건을 지정하여 특정 조건이 되면 AlertManager로 알람을 보낼 수 있음
prometheus.yml : 수집할 지표(metric)의 종류와 수집 주기등을 기입

# prometheus-config-map.yaml

apiVersion: v1

kind: ConfigMap

metadata:

labels:

namespace: monitoring

data:

prometheus.rules: |-

groups:

- name: container memory alert

rules:

- alert: container memory usage rate is very high( > 55%)

expr: sum(container_memory_working_set_bytes{pod!="", name=""})/ sum (kube_node_status_allocatable_memory_bytes) * 100 > 55

for: 1m

labels:

severity: fatal

annotations:

summary: High Memory Usage on

identifier: ""

description: " Memory Usage: "

- name: container CPU alert

rules:

- alert: container CPU usage rate is very high( > 10%)

expr: sum (rate (container_cpu_usage_seconds_total{pod!=""}[1m])) / sum (machine_cpu_cores) * 100 > 10

for: 1m

labels:

severity: fatal

annotations:

summary: High Cpu Usage

prometheus.yml: |-

global:

scrape_interval: 5s

evaluation_interval: 5s

rule_files:

- /etc/prometheus/prometheus.rules

alerting:

alertmanagers:

- scheme: http

static_configs:

- targets:

- "alertmanager.monitoring.svc:9093"

scrape_configs:

- job_name: 'kubernetes-apiservers'

kubernetes_sd_configs:

- role: endpoints

scheme: https

tls_config:

ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token

relabel_configs:

- source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]

action: keep

regex: default;kubernetes;https

- job_name: 'kubernetes-nodes'

scheme: https

tls_config:

ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token

kubernetes_sd_configs:

- role: node

relabel_configs:

- action: labelmap

regex: __meta_kubernetes_node_label_(.+)

- target_label: __address__

replacement: kubernetes.default.svc:443

- source_labels: [__meta_kubernetes_node_name]

regex: (.+)

target_label: __metrics_path__

replacement: /api/v1/nodes/${1}/proxy/metrics

- job_name: 'kubernetes-pods'

kubernetes_sd_configs:

- role: pod

relabel_configs:

- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]

action: keep

regex: true

- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]

action: replace

target_label: __metrics_path__

regex: (.+)

- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]

action: replace

regex: ([^:]+)(?::\d+)?;(\d+)

replacement: $1:$2

target_label: __address__

- action: labelmap

regex: __meta_kubernetes_pod_label_(.+)

- source_labels: [__meta_kubernetes_namespace]

action: replace

target_label: kubernetes_namespace

- source_labels: [__meta_kubernetes_pod_name]

action: replace

target_label: kubernetes_pod_name

- job_name: 'kube-state-metrics'

static_configs:

- targets: ['kube-state-metrics.kube-system.svc.cluster.local:8080']

- job_name: 'kubernetes-cadvisor'

scheme: https

tls_config:

ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token

kubernetes_sd_configs:

- role: node

relabel_configs:

- action: labelmap

regex: __meta_kubernetes_node_label_(.+)

- target_label: __address__

replacement: kubernetes.default.svc:443

- source_labels: [__meta_kubernetes_node_name]

regex: (.+)

target_label: __metrics_path__

replacement: /api/v1/nodes/${1}/proxy/metrics/cadvisor

- job_name: 'kubernetes-service-endpoints'

kubernetes_sd_configs:

- role: endpoints

relabel_configs:

- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]

action: keep

regex: true

- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]

action: replace

target_label: __scheme__

regex: (https?)

- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]

action: replace

target_label: __metrics_path__

regex: (.+)

- source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]

action: replace

target_label: __address__

regex: ([^:]+)(?::\d+)?;(\d+)

replacement: $1:$2

- action: labelmap

regex: __meta_kubernetes_service_label_(.+)

- source_labels: [__meta_kubernetes_namespace]

action: replace

target_label: kubernetes_namespace

- source_labels: [__meta_kubernetes_service_name]

action: replace

target_label: kubernetes_name

deployment

프로메테우스 이미지를 담은 pod을 담은 deployment controller

# prometheus-deployment.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

namespace: monitoring

spec:

replicas: 1

selector:

matchLabels:

app: prometheus-server

template:

metadata:

labels:

app: prometheus-server

spec:

containers:

- name: prometheus

image: prom/prometheus:latest

args:

- "--config.file=/etc/prometheus/prometheus.yml"

- "--storage.tsdb.path=/prometheus/"

ports:

- containerPort: 9090

volumeMounts:

- name: prometheus-config-volume

mountPath: /etc/prometheus/

- name: prometheus-storage-volume

mountPath: /prometheus/

volumes:

- name: prometheus-config-volume

configMap:

defaultMode: 420

- name: prometheus-storage-volume

emptyDir: {}

node exporter

프로메테우스가 수집하는 메트릭은 쿠버네티스에서 기본으로 제공하는 system metric만 수집하는게 아니라 그 외의 것들도 수집하기 때문에 수집역할을 하는 에이전트를 따로 두어야 합니다.

그 역할을 해주는게 node-exporter이고 각 노드에 하나씩 띄워야 하므로 DaemonSet으로 구성해주도록 합니다.

# prometheus-node-exporter.yaml

apiVersion: apps/v1

kind: DaemonSet

metadata:

namespace: monitoring

labels:

k8s-app: node-exporter

spec:

selector:

matchLabels:

k8s-app: node-exporter

template:

metadata:

labels:

k8s-app: node-exporter

spec:

containers:

- image: prom/node-exporter

ports:

- containerPort: 9100

protocol: TCP

---

apiVersion: v1

kind: Service

metadata:

labels:

k8s-app: node-exporter

namespace: kube-system

spec:

ports:

- name: http

port: 9100

nodePort: 31672

protocol: TCP

type: NodePort

selector:

k8s-app: node-exporter

Service

마지막으로 프로메테우스 pod을 외부로 노출시키는 서비스를 구성해줍니다.

# prometheus-svc.yaml

apiVersion: v1

kind: Service

metadata:

namespace: monitoring

annotations:

prometheus.io/scrape: 'true'

prometheus.io/port: '9090'

spec:

selector:

app: prometheus-server

type: NodePort

ports:

- port: 8080

targetPort: 9090

nodePort: 30003

배포

한 번에 배포 :

$ kubectl apply -f prometheus-cluster-role.yaml
$ kubectl apply -f prometheus-config-map.yaml
$ kubectl apply -f prometheus-deployment.yaml
$ kubectl apply -f prometheus-node-exporter.yaml
$ kubectl apply -f prometheus-svc.yaml

prometheus pod도 정상적으로 동작하고, 현재 실습환경은 노드가 2개이므로 node-exporter도 2개 뜬 것을 확인할 수 있습니다.

$ kubectl get pod -n monitoring

NAME                                     READY   STATUS    RESTARTS   AGE
node-exporter-99w2v                      1/1     Running   0          18s
node-exporter-f9q7f                      1/1     Running   0          18s
prometheus-deployment-7bcb5ff899-h4rb7   1/1     Running   0          65s

ip:30003로 접근해보면 prometheus의 web ui로 접근할 수 있습니다.

쿠버네티스의 다양한 metric들을 그래프로 확인할 수 있습니다.

상단 메뉴의 Status > Targets를 들어가보면, 프로메테우스가 현재 모니터링하고있는 타겟을 확인할 수 있습니다.

근데 이 중 kube-state-metrics가 (0/1 up)으로 아직 올라가지 않은 것으로 표시됩니다.

kube-state-metrics는 쿠버네티스 클러스터 내 오브젝트(예를들면 Pod)에 대한 지표정보를 생성하는 서비스입니다.

따라서 Pod 상태정보를 모니터링하기 위해서는 kube-state-metrics가 떠 있어야 합니다.

Kube State Metrics 배포

첫번째로 배포할 것은 ClusterRole과 ClusterRoleBinding입니다.

# kube-state-cluster-role.yaml

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

roleRef:

apiGroup: rbac.authorization.k8s.io

kind: ClusterRole

subjects:

- kind: ServiceAccount

namespace: kube-system

---

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRole

metadata:

rules:

- apiGroups:

- ""

resources: ["configmaps", "secrets", "nodes", "pods", "services", "resourcequotas", "replicationcontrollers", "limitranges", "persistentvolumeclaims", "persistentvolumes", "namespaces", "endpoints"]

verbs: ["list","watch"]

- apiGroups:

- extensions

resources: ["daemonsets", "deployments", "replicasets", "ingresses"]

verbs: ["list", "watch"]

- apiGroups:

- apps

resources: ["statefulsets", "daemonsets", "deployments", "replicasets"]

verbs: ["list", "watch"]

- apiGroups:

- batch

resources: ["cronjobs", "jobs"]

verbs: ["list", "watch"]

- apiGroups:

- autoscaling

resources: ["horizontalpodautoscalers"]

verbs: ["list", "watch"]

- apiGroups:

- authentication.k8s.io

resources: ["tokenreviews"]

verbs: ["create"]

- apiGroups:

- authorization.k8s.io

resources: ["subjectaccessreviews"]

verbs: ["create"]

- apiGroups:

- policy

resources: ["poddisruptionbudgets"]

verbs: ["list", "watch"]

- apiGroups:

- certificates.k8s.io

resources: ["certificatesigningrequests"]

verbs: ["list", "watch"]

- apiGroups:

- storage.k8s.io

resources: ["storageclasses", "volumeattachments"]

verbs: ["list", "watch"]

- apiGroups:

- admissionregistration.k8s.io

resources: ["mutatingwebhookconfigurations", "validatingwebhookconfigurations"]

verbs: ["list", "watch"]

- apiGroups:

- networking.k8s.io

resources: ["networkpolicies"]

verbs: ["list", "watch"]

위의 ClusterRole과 연동될 서비스어카운트를 생성해줍니다.

# kube-state-svcaccount.yaml

apiVersion: v1

kind: ServiceAccount

metadata:

namespace: kube-system

kube-state-metrics의 deployment도 구성해줍니다.

# kube-state-deployment.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

labels:

app: kube-state-metrics

namespace: kube-system

spec:

replicas: 1

selector:

matchLabels:

app: kube-state-metrics

template:

metadata:

labels:

app: kube-state-metrics

spec:

containers:

- image: quay.io/coreos/kube-state-metrics:v1.8.0

livenessProbe:

httpGet:

path: /healthz

port: 8080

initialDelaySeconds: 5

timeoutSeconds: 5

ports:

- containerPort: 8080

- containerPort: 8081

readinessProbe:

httpGet:

path: /

port: 8081

initialDelaySeconds: 5

timeoutSeconds: 5

nodeSelector:

kubernetes.io/os: linux

serviceAccountName: kube-state-metrics

마지막으로 kube-state-metrics의 서비스를 생성해주고 :

# kube-state-svc.yaml

apiVersion: v1

kind: Service

metadata:

labels:

app: kube-state-metrics

namespace: kube-system

spec:

clusterIP: None

ports:

- name: http-metrics

port: 8080

targetPort: http-metrics

- name: telemetry

port: 8081

targetPort: telemetry

selector:

app: kube-state-metrics

배포해주면 끝입니다.

$ kubectl apply -f kube-state-cluster-role.yaml
$ kubectl apply -f kube-state-deployment.yaml
$ kubectl apply -f kube-state-svcaccount.yaml
$ kubectl apply -f kube-state-svc.yaml

$ kubectl get pod -n kube-system

NAME READY STATUS RESTARTS AGE
...
kube-state-metrics-59bd4d9d-nbfrq 1/1 Running 0 50s

다시 프로메테우스의 웹으로 돌아가서 Target을 확인해보면 kube-state-metrics가 정상적으로 실행되고 있는 것을 확인할 수 있습니다.

2. Grafana 연동

그라파나는 수집 지표 정보를 분석 및 시각화 시키는 오픈소스 툴입니다.

주로 데이터를 시각화 하기 위한 대시보드로 주로 사용됩니다.

위에서 모니터링툴인 프로메테우스를 사용해 쿠버네티스의 metric들을 수집하는 것까지 했는데 이번엔 그라파나를 붙여서 프로메테우스의 데이터들을 시각화시켜보도록 하겠습니다.

먼저 그라파나 pod과 svc를 생성해줍니다.

# grafana.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

namespace: monitoring

spec:

replicas: 1

selector:

matchLabels:

app: grafana

template:

metadata:

labels:

app: grafana

spec:

containers:

- name: grafana

image: grafana/grafana:latest

ports:

- name: grafana

containerPort: 3000

env:

- name: GF_SERVER_HTTP_PORT

value: "3000"

- name: GF_AUTH_BASIC_ENABLED

value: "false"

- name: GF_AUTH_ANONYMOUS_ENABLED

value: "true"

- name: GF_AUTH_ANONYMOUS_ORG_ROLE

value: Admin

- name: GF_SERVER_ROOT_URL

value: /

---

apiVersion: v1

kind: Service

metadata:

namespace: monitoring

annotations:

prometheus.io/scrape: 'true'

prometheus.io/port: '3000'

spec:

selector:

app: grafana

type: NodePort

ports:

- port: 3000

targetPort: 3000

nodePort: 30004

$ kubectl apply -f grafana.yaml

$ kubectl get pod -n monitoring
NAME                                     READY   STATUS    RESTARTS   AGE
grafana-799c99855d-kxhkm                 1/1     Running   0          16s
node-exporter-99w2v                      1/1     Running   0          66m
node-exporter-f9q7f                      1/1     Running   0          66m
prometheus-deployment-7bcb5ff899-h4rb7   1/1     Running   0          67m

datasource 추가

ip:30004로 접속해보면 다음 화면을 보실 수 있습니다.

이제 그라파나와 프로메테우스를 연동시켜줄겁니다.

Add data source로 이동 :

시계열 데이터모델을 지원하는 여러 db를 확인할 수 있고, 우리는 프로메테우스를 사용할 것이니 프로메테우스를 선택 :

연동할 프로메테우스의 정보를 기입해주어야 합니다.

url은 서비스의 internal ip를 참고해서 적어주도록 합니다.

$ kubectl get svc -n monitoring

NAME                 TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
grafana              NodePort   10.101.152.163   <none>        3000:30004/TCP   6m43s
prometheus-service   NodePort   10.101.196.111   <none>        8080:30003/TCP   73m

Save & Test 버튼을 눌렀을 때 “Data source is working”메세지가 떠야합니다.

해당 메세지가 뜨면 연동은 성공!

연동을 했으니 프로메테우스의 데이터들을 시각화 해줄 dashboard를 만들어야합니다.

PromQL을 통해 사용자가 직접 원하는 대시보드를 만들수도있지만,

남이 만든것을 쉽게 가져다가 사용할수도 있습니다.

dashboard 추가

이번 실습에서는 PromQL을 작성하는게 아니라 다른사람이 만든 대시보드를 import해보도록 하겠습니다.

Grafana페이지로 이동 ->

Grafana > Dashboard로 이동해서 kubernetes를 검색합니다.

아무거나 마음에 드는것을 고르고 copy id를 눌러 대시보드의 id를 복사합니다.

그라파나 UI로 돌아와서 Import Dashboard로 이동해 복사한 번호를 붙여넣기 합니다.

항목입력해주고 Import해주면 예쁜 dashboard를 확인할 수 있습니다!

Deployment memory usage가 N/A로 뜨는 이유는 해당 dashboard의 PromQL이 저희의 prometheus와 맞지 않아서인데요… PromQL을 수정해주면 잘 뜬다고 하는데 한번 도전해보시기 바랍니다.

마치며

이렇게 쿠버네티스-프로메테우스-그라파나 가 완성되었습니다.

모니터링 구조의 기초 뼈대는 갖춰진 셈이니 이제 상황과 요구에 맞게 추가하시고 수정하시면 될 것 같습니다.

출처: <https://gruuuuu.github.io/cloud/monitoring-02/#>

Install Jira On Centos7

freedream — Wed, 19 Aug 2020 17:45:44 +0900

Install Jira On Centos7

JIRA is a issue tracking product, It is used for bug tracking, issue tracking, and project management system. In this blog, I will explain that how to install and configure latest version of JIRA on a Centos.This blog will help you to configure a JIRA with MySQL.

Requirement

JIRA require at least 1GB of RAM. If you not use a swap file, I recommend over 2GB of RAM.root privileges

JAVA software package is require for JIRA establishment. First. You need to install the java software package “java-1.8.0-openjdk-devel “on your centos server.

yum list java*jdk-devel

yum install -y java-1.8.0-openjdk-devel.x86_64

ls -l /usr/bin/javac

/usr/bin/javac -> /etc/alternatives/javac

ls -l /etc/alternatives/javac

/etc/alternatives/javac -> /usr/lib/jvm/java-1.8.0/bin/javac

alternatives --config java

alternatives --config javac

java -version

# Set JAVA_HOME environment variable.

echo "export JAVA_HOME=/usr/lib/jvm/java-1.8.0" >> ~/.bashrc

echo "PATH=$PATH:$JAVA_HOME/bin:$HOME/bin" >> ~/.bashrc

echo "export PATH" >> ~/.bashrc

source ~/.bashrc

env | grep JAVA

JAVA_HOME=/usr/lib/jvm/java-1.8.0

To install Jira on CentOS, please refer the following steps:

You need to download the latest JIRA Installer (.bin) file from the JIRA official page or given link to directory /opt

https://www.atlassian.com/ko/software/jira/update

cd /opt

wget https://www.atlassian.com/software/jira/downloads/binary/atlassian-jira-software-7.3.0-x64.bin

wget https://www.atlassian.com/software/jira/downloads/binary/atlassian-jira-software-8.5.6-x64.bin

After that, give the execute permission to .bin file and install JIRA.

#chmod +x atlassian-jira-software-7.3.0-x64.bin

#./atlassian-jira-software-7.3.0-x64.bin

Installation Directory: /opt/atlassian/jira

Home Directory: /var/atlassian/application-data/jira

HTTP Port: 8080

RMI Port: 8005

## Add the firewall rules

firewall-cmd --permanent --add-service=http

firewall-cmd --permanent --add-port={8080,8005}/tcp

firewall-cmd --reload

After you successful installation Jira, login URL is displayed and use it to login

http://server-ip:8080

or http://server-hostname:8080

To connect JIRA with MySQL, you need to copy the MySQL JDBC driver to your Jira server. Also, Copy the MySQL JDBC driver jar file to the JIRA installation directory /opt/atlassian/jira/lib/ . Also create a database and new user for JIRA.Also, you give the full permission for JIRA mysql user.

To configure the MySQL database

To create a database user for JIRA using following command:

#mysql -u root -p

CREATE DATABASE jiradb CHARACTER SET utf8 COLLATE utf8_bin;

grant all privileges on jiradb.* to 'jira'@'%' identified by 'syslint123!@#';

flush privileges;

exit

Copy the MySQL JDBC driver to your Jira server

After you installing the JIRA, you require MySQL Connector driver. You can download either the .tar.gz or the .zip file from official site. Otherwise, you can use the following command:

cd /opt

wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.35.tar.gz

# Extract the archive file

tar -zxvf mysql-connector-java-5.1.35.tar.gz

# Copy the MySQL JDBC driver jar file to the JIRA installation directory /opt/atlassian/jira/lib/

cd /opt/mysql-connector-java-5.1.35

cp mysql-connector-java-5.1.35-bin.jar /opt/atlassian/jira/lib/

To restart Jira Service

cd /opt/atlassian/jira/bin/

./shutdown.sh

./startup.sh

https://syslint.com/blog/tutorial/how-to-install-jira-on-centos/

https://confluence.atlassian.com/adminjiraserver071/installing-jira-applications-on-linux-802592173.html

https://confluence.atlassian.com/jirasoftware/jira-software-8-5-x-release-notes-975014654.html#JiraSoftware8.5.xreleasenotes-check

Install Artifactory on CentOS 7

freedream — Wed, 19 Aug 2020 17:43:43 +0900

Installing JFrog Artifactory on CentOS / RHEL 7

Artifactory is a universal repository manager created by JFrog. A repository manager is software application designed to manage binary components related to an application. Being a universal repository manager, Artifactory supports all major packaging formats like Apache Maven, Gradle, Docker and many more.

In DevOps environments, Artifactory is used by build tools such as Apache Ant, Maven and Gradle to serve as the local repository for their respective artifacts. Although using Artifactory is not mandatory in DevOps environment, but it can be used to accelerate the CI/CD process.

JFrog Artifactory is a commercial product. However, there is an OSS (Open Source Software) version is available with limited features. We will use this OSS version in our article. We will install JFrog Artifactory OSS on CentOS / RHEL 7 server and deploy and use MariaDB database as the backend database for Artifactory.

For more information on DevOps best practices and CI/CD implementation, please read Continuous Delivery for Java Apps by Leanpub.

This Article Provides:

1. System Specification

We have used a CentOS 7 virtual machine with following specification for this article.

Hostname - artifactory-01.example.com
IP Address - 192.168.116.130/24
Operating System - CentOS 7.6

2. Installing Java Development Kit on CentOS 7

JFrog Artifactory requires Java Development Kit (JDK) 8 update 45 or above.

In CentOS 7, the OpenJDK (an open-source Java Development Kit) can be installed using yum command. However, if you are using a Red Hat Enterprise Linux (RHEL) 7 then you have to configure a Local yum Repository and then you will be able to install OpenJDK using yum command.

yum list java*jdk-devel

yum install java-1.8.0-openjdk-devel.x86_64

# ln -s /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64 /usr/lib/jvm/java-1.8.0

ls -l /usr/bin/javac

/usr/bin/javac -> /etc/alternatives/javac

ls -l /etc/alternatives/javac

/etc/alternatives/javac -> /usr/lib/jvm/java-1.8.0/bin/javac

alternatives --config java

alternatives --config javac

java -version

Set JAVA_HOME environment variable.

echo "export JAVA_HOME=/usr/lib/jvm/java-1.8.0" >> ~/.bashrc

echo "PATH=$PATH:$JAVA_HOME/bin:$HOME/bin" >> ~/.bashrc

echo "export PATH" >> ~/.bashrc

source ~/.bashrc

env | grep JAVA

JAVA_HOME=/usr/lib/jvm/java-1.8.0

3. Installing JFrog Artifactory OSS on CentOS 7

Install preqrequisite packages using yum command.

yum install -y net-tools rsync

Download latest JFrog Artifactory OSS RPM and copy it to the home directory of root user.

https://jfrog.com/open-source/#artifactory

wget https://bintray.com/jfrog/artifactory-rpms/rpm -O /etc/yum.repos.d/bintray-jfrog-artifactory-oss-rpms.repo;

yum install jfrog-artifactory-oss

# vi /etc/yum.repos.d/bintray-jfrog-artifactory-oss-rpms.repo

[bintray--jfrog-artifactory-rpms]

name=bintray--jfrog-artifactory-rpms

baseurl=https://jfrog.bintray.com/artifactory-rpms

gpgcheck=0

repo_gpgcheck=0

enabled=1

rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

# Install Jfrog Artifactory RPM

yum localinstall -y jfrog-artifactory-oss-7.4.3.rpm

# download tar.gz

https://api.bintray.com/content/jfrog/artifactory/org/artifactory/oss/jfrog-artifactory-oss/$latest/jfrog-artifactory-oss-$latest-linux.tar.gz?bt_package=jfrog-artifactory-oss

mkdir -p /opt/jfrog

tar -xvf jfrog-artifactory-oss-7.4.3-linux.tar.gz -O /opt/jfrog/

ln -s /opt/jfrog/artifactory-oss-7.4.3 /opt/jfrog/artifactory

chown -R artifactory:artifactory /opt/jfrog/artifactory

Set ARTIFACTORY_HOME environment variable.

echo "export ARTIFACTORY_HOME=/opt/jfrog/artifactory" >> ~/.bashrc

source ~/.bashrc

env | grep ARTIFACTORY_HOME

JFrog Artifactory has been installed on CentOS 7.

4. Installing MariaDB Database on CentOS 7

It is recommended to use JFrog Artifactory with an external database. Being a hardcore FOSS (Fan of Open Source Software), we selected MariaDB Database for this purpose.

Install MariaDB Server from CentOS yum repository.

yum install -y mariadb-server

Start and Enable MariaDB Service.

systemctl start mariadb

systemctl enable mariadb

systemctl status mariadb

cat /usr/lib/systemd/system/mariadb.service

mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB

SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!

In order to log into MariaDB to secure it, we'll need the current

password for the root user. If you've just installed MariaDB, and

you haven't set the root password yet, the password will be blank,

so you should just press enter here.

Enter current password for root (enter for none):

OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MariaDB

root user without the proper authorisation.

Set root password? [Y/n] Y

New password:

Re-enter new password:

Password updated successfully!

Reloading privilege tables..

... Success!

By default, a MariaDB installation has an anonymous user, allowing anyone

to log into MariaDB without having to have a user account created for

them. This is intended only for testing, and to make the installation

go a bit smoother. You should remove them before moving into a

production environment.

Remove anonymous users? [Y/n] Y

... Success!

Normally, root should only be allowed to connect from 'localhost'. This

ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] Y

... Success!

By default, MariaDB comes with a database named 'test' that anyone can

access. This is also intended only for testing, and should be removed

before moving into a production environment.

Remove test database and access to it? [Y/n] Y

- Dropping test database...

... Success!

- Removing privileges on test database...

... Success!

Reloading the privilege tables will ensure that all changes made so far

will take effect immediately.

Reload privilege tables now? [Y/n] Y

... Success!

Cleaning up...

All done! If you've completed all of the above steps, your MariaDB

installation should now be secure.

Thanks for using MariaDB!

cat /opt/jfrog/artifactory/var/etc/system.yaml

cat /opt/jfrog/artifactory/var/etc/security/master.key

cat /opt/jfrog/artifactory/var/etc/security/join.key

systemctl restart mariadb

systemctl status mariadb

● Create and configure Artifactory database

# Edit artifactory.service and add the user and group who own the service.

/opt/jfrog/artifactory/app/bin/installService.sh

vi /usr/lib/systemd/system/artifactory.service

[Unit]

Description=Artifactory service

After=network.target

[Service]

Type=forking

User=artifactory

Group=artifactory

GuessMainPID=yes

PIDFile=/var/run/artifactory.pid

ExecStart=/opt/jfrog/artifactory/app/bin/artifactoryManage.sh start

ExecStop=/opt/jfrog/artifactory/app/bin/artifactoryManage.sh stop

LimitNOFILE=65536

LimitNPROC=8192

Restart=always

RestartSec=60

[Install]

WantedBy=multi-user.target

Alias=artifactory.service

Set open file limits for the artifactory user.

echo "artifactory soft nofile 32000" >> /etc/security/limits.conf

echo "artifactory hard nofile 32000" >> /etc/security/limits.conf

# start artifactory service

systemctl daemon-reload

systemctl restart artifactory

systemctl enable artifactory

systemctl status artifactory

tail -f /opt/jfrog/artifactory/var/log/console.log

# SELinux 활성화 시

chmod -R +r /opt/jfrog/artifactory/app/artifactory/tomcat/webapps

restorecon -vR /opt/jfrog/artifactory/app/artifactory/tomcat/webapps

# SELinux 비활성화(disabled or permissive)

setenforce 0

sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

# firewall setting

firewall-cmd --permanent --add-port=8081/tcp

firewall-cmd --reload

Browse URL http://artifactory-01.example.com:8081 from a client browser. ( default: admin / password )

Click on Next.

Set the password for default admin user of Artifactory and click on Next.

If this server access the Internet through a proxy, then configure it here or click on Skip.

Select the package types for which you want to create repositories. The default repositories for the selected package types will be created by the Artifactory. Or click on Skip.

The initial configuration of Artifactory has been completed. Click on Finish.

You have reached at the Dashboard of Jfrog Artifactory Web UI. This dashboard is enriched with links to guides and documentation related to JFrog Artifactory. You can browse these links and start using JFrog Artifactory accordingly.

JFrog Artifactory has been successfully installed and configured on CentOS / RHEL 7 server.

https://www.centlinux.com/2019/01/install-jfrog-artifactory-on-centos-rhel-7.html

https://www.jfrog.com/confluence/display/JFROG/Installing+Artifactory

Install SonarQube on CentOS 7

freedream — Wed, 19 Aug 2020 17:39:21 +0900

----------------------------------------------------------

- Install SonarQube on CentOS 7

----------------------------------------------------------

sonarQube is an open-source platform for continuous inspection of code quality. It is used to perform automatic reviews with static analysis of code to detect bugs, code smells, and security vulnerabilities on more than 20 programming languages.

Here we are going to install and configure SonarQube 7.9.x LTS with Oracle JAVA 11, PostgreSQL 10.x, Nginx, and Let’s Encrypt certificates.

Execute the following commands using the root user.

1. Update System

----------------------------------------------------------

yum update

2. Disable SELinux

----------------------------------------------------------

vim /etc/sysconfig/selinux

Change "SELINUX=enforcing" to "SELINUX=disabled".

setenforce 0

sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config

3. Install Base Packages

----------------------------------------------------------

yum install wget vim zip unzip -y

hostnamectl set-hostname $DOMAIN_NAME

reboot

4. Prerequisite

----------------------------------------------------------

You can check the official document for complete information.

• Java (Oracle JRE 11 or OpenJDK 11)

• PostgreSQL 10 or 9.3–9.6

Hardware Requirements

• Server with 2GB or plus RAM

• Systems setting for Linux

vm.max_map_count is greater or equals to 262144

fs.file-max is greater or equals to 65536

the user running SonarQube can open at least 65536 file descriptors

the user running SonarQube can open at least 4096 threads

5. Add System settings

----------------------------------------------------------

vim /etc/sysctl.conf

vm.max_map_count=262144

fs.file-max=65536

sysctl -w vm.max_map_count=262144

sysctl -w fs.file-max=65536

sysctl -p

vim /etc/security/limits.conf

session required pam_limits.so

root hard nofile 65535

root soft nofile 65535

root hard nproc 65535

root soft nproc 65535

# elasticsearch

sonar hard nofile 65535

sonar soft nofile 65535

sonar hard nproc 65535

sonar soft nproc 65535

sonar hard memlock unlimited

sonar soft memlock unlimited

vi /etc/security/limits.d/20-nproc.conf

reboot

ulimit -Sa

ulimit -Ha

6. Install OpenJDK 11

----------------------------------------------------------

yum list java*jdk-devel

yum install java-11-openjdk-devel.x86_64

ls -l /usr/bin/javac

/usr/bin/javac -> /etc/alternatives/javac

ls -l /etc/alternatives/javac

/etc/alternatives/javac -> /usr/lib/jvm/java-11-openjdk-11.0.7.10-4.el7_8.x86_64/bin/javac

alternatives --config java

alternatives --config javac

java -version

# Set JAVA_HOME environment variable.

echo "export JAVA_HOME=/usr/lib/jvm/java-11" >> ~/.bashrc

echo "PATH=$PATH:$JAVA_HOME/bin:$HOME/bin" >> ~/.bashrc

echo "export PATH" >> ~/.bashrc

source ~/.bashrc

env | grep JAVA

JAVA_HOME=/usr/lib/jvm/java-11

7. Install PostgreSQL 10

----------------------------------------------------------

1) PostgreSQL Yum Repository download / Install

https://www.postgresql.org/download/linux/redhat/

wget https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm

yum localinstall pgdg-redhat-repo-latest.noarch.rpm

2) Install server

yum install postgresql10-server postgresql10-contrib

3)Initialize the database

/usr/pgsql-10/bin/postgresql-10-setup initdb

4) Modify pg_hba.conf file

change "peer" to "trust" and "idnet" to "md5".

vim /var/lib/pgsql/10/data/pg_hba.conf

----------------------------------------------------------

# TYPE DATABASE USER ADDRESS METHOD

# "local" is for Unix domain socket connections only

local all all peer => trust

# IPv4 local connections:

host all all 127.0.0.1/32 ident => md5

# IPv6 local connections:

host all all ::1/128 ident => md5

# Allow replication connections from localhost, by a user with the

# replication privilege.

local replication all peer

host replication all 127.0.0.1/32 ident

host replication all ::1/128 ident

----------------------------------------------------------

5) To start service and set on boot, enable PostgreSQL on system boot

systemctl start postgresql-10

systemctl enable postgresql-10

systemctl status postgresql-10

6) create Postgres user & database

# Change the default password

passwd postgres

su - postgres

# Create a new user

createuser sonar

# Switch to PostgreSQL shell

psql

# Set a password for the newly created user for the SonarQube database

ALTER USER sonar WITH ENCRYPTED password 'd98ffW@123?Q';

# CREATE USER sonar WITH ENCRYPTED PASSWORD 'd98ffW@123?Q';

# Create a new database for the PostgreSQL database

CREATE DATABASE sonar OWNER sonar;

ALTER ROLE sonar WITH createdb;

GRANT ALL PRIVILEGES ON DATABASE sonar TO sonar;

# Exit from the psql shell.

# Exit from the "postgres" user.

exit

systemctl restart postgresql-10

systemctl status postgresql-10

8. Download and configure SonarQube

----------------------------------------------------------

1) Download Latest LTS version

https://www.sonarqube.org/downloads/

#wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-8.3.1.34397.zip --no-check-certificate

wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-7.9.3.zip --no-check-certificate

unzip sonarqube-7.9.3.zip -d /opt

mv /opt/sonarqube-7.9.3 /opt/sonarqube

2) Modify sonar.properties

vim /opt/sonarqube/conf/sonar.properties

sonar.jdbc.username=sonar

sonar.jdbc.password=d98ffW@123?Q

sonar.jdbc.url=jdbc:postgresql://localhost/sonar

sonar.web.javaAdditionalOpts=-server

sonar.web.host=127.0.0.1

sonar.web.port=9000

sonar.web.http.maxThreads=50

sonar.web.http.minThreads=5

sonar.web.http.acceptCount=25

sonar.web.javaOpts=-server -Xms512m -Xmx512m -XX:+HeapDumpOnOutOfMemoryError

sonar.search.javaOpts=-server -Xms512m -Xmx512m -XX:+HeapDumpOnOutOfMemoryError

# Configuring the Elasticsearch storage path

sonar.path.data=/var/sonarqube/data

sonar.path.temp=/var/sonarqube/temp

# log setting

sonar.log.level=INFO

sonar.log.level.app=INFO

sonar.log.level.web=INFO

sonar.log.level.ce=INFO

sonar.log.level.es=INFO

sonar.path.logs=logs

sonar.log.rollingPolicy=time:yyyy-MM-dd

sonar.log.maxFiles=7

sonar.web.accessLogs.enable=true

3) Create a user for sonar

groupadd -g 1005 -r sonar

useradd -c "sonar" -u 1005 -g sonar -s /bin/bash -r -p password sonar

4) Modify folder permissions

chown -R sonar:sonar /opt/sonarqube

# Create folders and grant permission

mkdir -p /var/sonarqube/data

mkdir -p /var/sonarqube/temp

chown -R sonar:sonar /var/sonarqube

5) Setting up Sonarqube as a service

vim /etc/systemd/system/sonarqube.service

[Unit]

Description=SonarQube service

After=syslog.target network.target

[Service]

Type=forking

User=sonar

Group=sonar

ExecStart=/opt/sonarqube/bin/linux-x86-64/sonar.sh start

ExecStop=/opt/sonarqube/bin/linux-x86-64/sonar.sh stop

ExecReload=/opt/sonarqube/bin/linux-x86-64/sonar.sh restart

LimitNOFILE=65536

LimitNPROC=8192

Restart=on-failure

TimeoutStartSec=60

[Install]

WantedBy=multi-user.target

# Reload daemon and enable sonar on system boot

systemctl daemon-reload

systemctl restart sonarqube

systemctl enable sonarqube

systemctl status sonarqube

6) logfile location

cd /opt/sonarqube/logs/

• SonarQube service log

tail -f /opt/sonarqube/logs/sonar.log

• Web Server Logs

tail -f /opt/sonarqube/logs/web.log

• ElasticSearch logs

tail -f /opt/sonarqube/logs/es.log

• Compute Engine logs

tail -f /opt/sonarqube/logs/ce.log

9. elasticsearch setting

----------------------------------------------------------

/opt/sonarqube/elasticsearch

vi /opt/sonarqube/elasticsearch/bin/elasticsearch-env

# JDK 1.8 추가

JAVA_HOME=/usr/lib/jvm/java-1.8.0

/opt/sonarqube/elasticsearch/config

vi /opt/sonarqube/elasticsearch/config/jvm.options

# JVM heap size setting

-Xms1g -Xmx4g

vi /opt/sonarqube/elasticsearch/config/elasticsearch.yml

cluster.name: my-application

node.name: node-1

node.attr.rack: r1

path.data: /var/sonarqube/data

path.logs: /var/sonarqube/logs

bootstrap.memory_lock: true

network.host: ip

http.port: 9200

discovery.zen.ping.unicast.hosts: ["127.0.0.1", "ip"]

vi /opt/sonarqube/elasticsearch/config/log4j2.properties

10. Configure reverse proxy

----------------------------------------------------------

# Install Nginx, start service, and enable on system boot

cat << EOF > /etc/yum.repos.d/nginx.repo

[nginx]

name=Nginx Repository \$basearch - Archive

baseurl=http://nginx.org/packages/centos/\$releasever/\$basearch/

enabled=1

gpgcheck=1

gpgkey=https://nginx.org/keys/nginx_signing.key

EOF

yum install -y nginx

systemctl start nginx

systemctl enable nginx

systemctl status nginx

11. Configure SSL

----------------------------------------------------------

# Enable epel repo

yum install – y epel-release

# create certificate

=> SSL 구성 추가

vim /etc/nginx/nginx.conf

# add the following contents to a Location Blocks.

location / {

proxy_pass "http://127.0.0.1:9000";

proxy_http_version 1.1;

proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection 'upgrade';

proxy_set_header Host $host;

proxy_cache_bypass $http_upgrade;

}

# Check syntax & Restart Nginx

nginx -t

systemctl restart nginx

systemctl status nginx

12. DNS

----------------------------------------------------------

Then go to your DNS manager and add A record for your sonar server.

A Domain Name Server IP

13. Modify Firewall Rules

----------------------------------------------------------

firewall-cmd --zone=public --permanent --add-service=https

firewall-cmd --zone=public --add-port=9000/tcp --permanent

firewall-cmd --zone=public --add-port=443/tcp --permanent

firewall-cmd --reload

# Firewall Rule HTTPS

if you need to open sonar for specific IP, run the below command

firewall-cmd --permanent --zone=public --add-rich-rule='

rule family="ipv4"

source address="122.43.8.188/32"

port protocol="tcp" port="443" accept'

firewall-cmd --reload

14. Browse Sonarqube

----------------------------------------------------------

Go to your browser and type your domain name.

https://sonar.example.com/

http://sonar.example.com:9000

The default username and password is "admin"

https://www.fosslinux.com/24429/how-to-install-and-configure-sonarqube-on-centos-7.htm

Install Nexus OSS on Centos 7

freedream — Wed, 19 Aug 2020 17:37:53 +0900

1. system Requirements

------------------------------------------------------------------

● cpu

Minimum CPUs: 4, Recommended CPUs: 8+

● memory

JVM Heap JVM Direct Host Physical RAM

------------------------------------------------------------------

Minimum ( default ) 2703MB 2703MB 8GB

Maximum 4GB (host physical RAM * 2/3) - JVM max heap no limit

● disk space

formats like Docker and Maven can use very large amounts of storage (500Gb easily).

2. yum update & install OpenJDK

------------------------------------------------------------------

yum update -y

yum -y install wget vim

# Install OpenJDK

yum list java*jdk-devel

yum install -y java-1.8.0-openjdk-devel.x86_64

java-1.8.0-openjdk-devel-1.8.0.252.b09-2.el7_8.x86_64.rpm

#ln -s /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64 /usr/lib/jvm/java-1.8.0

ls -l /usr/bin/javac

/usr/bin/javac -> /etc/alternatives/javac

ls -l /etc/alternatives/javac

/etc/alternatives/javac -> /usr/lib/jvm/java-1.8.0/bin/javac

alternatives --config java

alternatives --config javac

java -version

# Set JAVA_HOME environment variable.

echo "export JAVA_HOME=/usr/lib/jvm/java-1.8.0" >> ~/.bashrc

echo "PATH=$PATH:$JAVA_HOME/bin:$HOME/bin" >> ~/.bashrc

echo "export PATH" >> ~/.bashrc

source ~/.bashrc

env | grep JAVA

JAVA_HOME=/usr/lib/jvm/java-1.8.0

3. Download and extract nexus

------------------------------------------------------------------

wget -O nexus.tar.gz https://download.sonatype.com/nexus/3/latest-unix.tar.gz

tar xvf nexus.tar.gz -C /opt

/opt/nexus-3.23.0-03

/opt/sonatype-work

ln -s /opt/nexus-3.23.0-03/ /opt/nexus

ln -s /opt/nexus/bin/nexus /etc/init.d/nexus

# nexus 2 download

wget -O nexus2.tar.gz https://download.sonatype.com/nexus/oss/nexus-latest-bundle.tar.gz

4. Setup nexus user

------------------------------------------------------------------

# Create nexus user

groupadd -g 1002 -r nexus

useradd -c "nexus3 oss" -u 1002 -g nexus -s /bin/bash -r -p password nexus

chown -R nexus:nexus /opt/nexus

chown -R nexus:nexus /opt/sonatype-work

5. Running as nexus user

------------------------------------------------------------------

vim /opt/nexus/bin/nexus.rc

run_as_user="nexus"

#VM options

vim /opt/nexus/bin/nexus.vmoptions

-Xms2703m

-Xmx2703m

-XX:MaxDirectMemorySize=2703m

-XX:+UnlockDiagnosticVMOptions

-XX:+LogVMOutput

-XX:LogFile=../sonatype-work/nexus3/log/jvm.log

-XX:-OmitStackTraceInFastThrow

-Djava.net.preferIPv4Stack=true

-Dkaraf.home=.

-Dkaraf.base=.

-Dkaraf.etc=etc/karaf

-Djava.util.logging.config.file=etc/karaf/java.util.logging.properties

-Dkaraf.data=../sonatype-work/nexus3

-Dkaraf.log=../sonatype-work/nexus3/log

-Djava.io.tmpdir=../sonatype-work/nexus3/tmp

-Dkaraf.startLocalConsole=false

6. Running nexus as Linux service

------------------------------------------------------------------

#ln -s /opt/nexus/bin/nexus /etc/init.d/nexus

systemctl enable nexus

Executing /sbin/chkconfig nexus ons

#cd /etc/init.d

#chkconfig --add nexus

#chkconfig --levels 345 nexus on

#service nexus start

# vim /etc/security/limits.conf

nexus - nofile 65536

# vim /etc/systemd/system/nexus.service

[Unit]

Description=nexus service

After=network.target

[Service]

Type=forking

LimitNOFILE=65536

User=nexus

Group=nexus

ExecStart=/opt/nexus/bin/nexus start

ExecStop=/opt/nexus/bin/nexus stop

User=nexus

Restart=on-abort

TimeoutSec=600

[Install]

WantedBy=multi-user.target

7. Starting nexus

------------------------------------------------------------------

# open firewall

firewall-cmd --permanent --zone=public --add-port=22/tcp

firewall-cmd --permanent --zone=public --add-port=8081/tcp

firewall-cmd --reload

firewall-cmd --permanent --list-all

systemctl daemon-reload

systemctl enable nexus

systemctl restart nexus

systemctl status nexus

tail -f /opt/sonatype-work/nexus3/log/nexus.log

# console login

http://localhost:8081

Default username is admin / admin123

# default admin password

cat /opt/sonatype-work/nexus3/admin.password

8. Running behind reverse proxy

------------------------------------------------------------------

yum install -y httpd

# vim /opt/sonatype-work/nexus3/etc/nexus.properties

nexus-context-path=/nexus

# vim /etc/httpd/conf.d/nexus.conf

ProxyRequests Off

ProxyPreserveHost On

ServerName centos.dev

ServerAdmin admin@centos.dev

ProxyPass /nexus http://localhost:8081/nexus

ProxyPassReverse /nexus http://localhost:8081/nexus

ErrorLog /var/log/nexus/error.log

CustomLog /var/log/nexus/access.log common

</VirtualHost>

9. ErrorLog and CustomLog directives configuration

------------------------------------------------------------------

mkdir /var/log/nexus

apachectl configtest

Syntax OK

systemctl httpd restart

10. Maven Artifacts Uploader

------------------------------------------------------------------

This is a friendly command line tool for uploading a directory of artifacts to Nexus 3.x repository

1) Installation

● Clone package

git clone https://github.com/ronbadur/maven-artifacts-uploader.git

cd maven-artifacts-uploader

mvn install

● conf/config.properties

=> repository url, repositroy id

● PATH enviroment

echo "export MVN-UPLOAD=/opt/maven-artifacts-uploader" >> ~/.bashrc

echo "PATH=$PATH:$MVN-UPLOAD/bin:$HOME/bin" >> ~/.bashrc

echo "export PATH" >> ~/.bashrc

● settings.xml

<id>nexus</id>

<username>admin</username>

<password>admin123</password>

</server>

</servers>

2) upload all the artifacts in specific directory

#mvnUploader -d path/to/your/artifacts

java -jar ./lib\nexus-uploader.jar -d /opt/mvn-repo

# To get all the options that available

mvnUploader -h

11. upload files into Nexus 3

------------------------------------------------------------------

curl -v -u admin:admin123 --upload-file pom.xml http://localhost:8081/repository/maven-releases/org/foo/1.0/foo-1.0.pom

# Example without a pom file

mvn deploy:deploy-file -DgroupId=com.somecompany -DartifactId=project -Dversion=1.0.0 -DgeneratePom=true -Dpackaging=jar -DrepositoryId=nexus -Durl=http://localhost:8081/repository/maven-releases -Dfile=target/project-1.0.0.jar

# With a pom file

mvn deploy:deploy-file -DgeneratePom=false -DrepositoryId=nexus -Durl=http://localhost:8081/repository/maven-releases -DpomFile=pom.xml -Dfile=target/project-1.0.0.jar

12. Gradle을 이용한 Nexus 저장소 Upload

------------------------------------------------------------------

Nexus 배포를 위한 uploadArchives 스크립트를 작성

# vi build.gradle

apply plugin: 'maven'

group = "paas.test"

version = '0.0.1-SNAPSHOT'

project.group = 'paas.test'

uploadArchives {

repositories {

mavenDeployer {

repository(url: 'http://your.nexus.repository.com/nexus/content/repositories/releases/') {

authentication(userName: '계정명', password: '비밀번호')

}

snapshotRepository(url: 'http://your.nexus.repository.com/nexus/content/repositories/snapshot/') {

authentication(userName: '계정명', password: '비밀번호')

}

=> eclipse에서 uploadArchives를 실행하여 Nexus에 업로드

https://help.sonatype.com/repomanager3/installation

https://drmanalo.github.io/blog/2017/installing-nexus-centos7.html

https://devopscube.com/how-to-install-latest-sonatype-nexus-3-on-linux/

https://help.sonatype.com/repomanager2/download

https://github.com/ronbadur/maven-artifacts-uploader

https://support.sonatype.com/hc/en-us/articles/115006744008-How-can-I-programmatically-upload-files-into-Nexus-3-

https://kingbbode.tistory.com/37

Install Harbor 2.0 on CentOS 7

freedream — Wed, 19 Aug 2020 17:36:15 +0900

------------------------------------------------------------

-- 1. Install Python

------------------------------------------------------------

1) Install Python Dependencies

yum -y groupinstall "Development Tools"

yum -y install openssl-devel bzip2-devel libffi-devel

gcc --version

gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)

2) Download latest Python 3.8 Archive

yum -y install wget

wget https://www.python.org/ftp/python/3.8.3/Python-3.8.3.tgz

tar xvf Python-3.8.3.tgz -O /opt

cd /opt/Python-3.8.3

3) Install Python 3.8

./configure --enable-optimizations

make altinstall

4) Check Python 3.8

python3.8 --version

pip3.8 --version

------------------------------------------------------------

-- 2. Install Docker Engine

------------------------------------------------------------

1) Uninstall old versions:

yum remove docker docker-common docker-selinux docker-engine

2) Install Prereqs

yum install -y yum-utils device-mapper-persistent-data lvm2

3) Setup stable repo

yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

4) Install Docker CE

yum -y install docker-ce docker-ce-cli containerd.io

5) If you get dependency errors, the run

yum install -y --setopt=obsoletes=0 docker-ce docker-ce-selinux

6) Start and enable docker service

systemctl start docker && systemctl enable docker

systemctl enable --now docker

systemctl is-active docker

systemctl is-enabled docker

newgrp docker

docker version

usermod -aG docker root

# Create Harbor User

groupadd -g 1001 -r harbor

useradd -c "Harbor" -u 1001 -g harbor -s /bin/bash -r -p password harbor

usermod -aG docker harbor

------------------------------------------------------------

-- 3. Install Docker Compose

------------------------------------------------------------

● Run this command to download the current stable release of Docker Compose

curl -L "https://github.com/docker/compose/releases/download/1.25.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

chmod +x /usr/local/bin/docker-compose

ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

● install command completion for the bash

curl -L https://raw.githubusercontent.com/docker/compose/1.25.5/contrib/completion/bash/docker-compose -o /etc/bash_completion.d/docker-compose

# Source the file or re-login to enjoy completion feature.

source /etc/bash_completion.d/docker-compose

● Test the installation.

docker-compose --version

docker-compose version 1.25.5, build 8a1c60f6

● Upgrading

docker-compose 1.5.0

docker container rm -f -v myapp_web_1 myapp_db_1 ...

● Uninstallation

# To uninstall Docker Compose if you installed using curl

rm /usr/local/bin/docker-compose

# To uninstall Docker Compose if you installed using pip

pip uninstall docker-compose

● docker image pull & tag

# harbor docker image pull

-----------------------------------------------------

docker pull goharbor/harbor-log:v2.0.0

docker pull goharbor/registry-photon:v2.0.0

docker pull goharbor/harbor-registryctl:v2.0.0

docker pull goharbor/harbor-db:v2.0.0

docker pull goharbor/harbor-core:v2.0.0

docker pull goharbor/harbor-portal:v2.0.0

docker pull goharbor/harbor-jobservice:v2.0.0

docker pull goharbor/redis-photon:v2.0.0

docker pull goharbor/nginx-photon:v2.0.0

docker pull goharbor/notary-server-photon:v2.0.0

docker pull goharbor/notary-signer-photon:v2.0.0

docker pull goharbor/clair-photon:v2.0.0

docker pull goharbor/clair-adapter-photon:v2.0.0

docker pull goharbor/chartmuseum-photon:v2.0.0

# harbor docker image save

-----------------------------------------------------

docker save -o harbor-images.tar \

goharbor/harbor-log:v2.0.0 \

goharbor/registry-photon:v2.0.0 \

goharbor/harbor-registryctl:v2.0.0 \

goharbor/harbor-db:v2.0.0 \

goharbor/harbor-core:v2.0.0 \

goharbor/harbor-portal:v2.0.0 \

goharbor/harbor-jobservice:v2.0.0 \

goharbor/redis-photon:v2.0.0 \

goharbor/nginx-photon:v2.0.0 \

goharbor/notary-server-photon:v2.0.0 \

goharbor/notary-signer-photon:v2.0.0 \

goharbor/clair-photon:v2.0.0 \

goharbor/clair-adapter-photon:v2.0.0 \

goharbor/chartmuseum-photon:v2.0.0

# harbor docker image load

-----------------------------------------------------

docker load -i harbor-images.tar

# harbor docker image tag

-----------------------------------------------------

docker tag goharbor/harbor-log:v2.0.0 registry.test.paas/library/harbor-log:v2.0.0

docker tag goharbor/registry-photon:v2.0.0 registry.test.paas/library/registry-photon:v2.0.0

docker tag goharbor/harbor-registryctl:v2.0.0 registry.test.paas/library/harbor-registryctl:v2.0.0

docker tag goharbor/harbor-db:v2.0.0 registry.test.paas/library/harbor-db:v2.0.0

docker tag goharbor/harbor-core:v2.0.0 registry.test.paas/library/harbor-core:v2.0.0

docker tag goharbor/harbor-portal:v2.0.0 registry.test.paas/library/harbor-portal:v2.0.0

docker tag goharbor/harbor-jobservice:v2.0.0 registry.test.paas/library/harbor-jobservice:v2.0.0

docker tag goharbor/redis-photon:v2.0.0 registry.test.paas/library/redis-photon:v2.0.0

docker tag goharbor/nginx-photon:v2.0.0 registry.test.paas/library/nginx-photon:v2.0.0

docker tag goharbor/notary-server-photon:v2.0.0 registry.test.paas/library/notary-server-photon:v2.0.0

docker tag goharbor/notary-signer-photon:v2.0.0 registry.test.paas/library/notary-signer-photon:v2.0.0

docker tag goharbor/clair-photon:v2.0.0 registry.test.paas/library/clair-photon:v2.0.0

docker tag goharbor/clair-adapter-photon:v2.0.0 registry.test.paas/library/clair-adapter-photon:v2.0.0

docker tag goharbor/chartmuseum-photon:v2.0.0 registry.test.paas/library/chartmuseum-photon:v2.0.0

# harbor docker image push to registry

-----------------------------------------------------

docker push registry.test.paas/library/harbor-log:v2.0.0

docker push registry.test.paas/library/registry-photon:v2.0.0

docker push registry.test.paas/library/harbor-registryctl:v2.0.0

docker push registry.test.paas/library/harbor-db:v2.0.0

docker push registry.test.paas/library/harbor-core:v2.0.0

docker push registry.test.paas/library/harbor-portal:v2.0.0

docker push registry.test.paas/library/harbor-jobservice:v2.0.0

docker push registry.test.paas/library/redis-photon:v2.0.0

docker push registry.test.paas/library/nginx-photon:v2.0.0

docker push registry.test.paas/library/notary-server-photon:v2.0.0

docker push registry.test.paas/library/notary-signer-photon:v2.0.0

docker push registry.test.paas/library/clair-photon:v2.0.0

docker push registry.test.paas/library/clair-adapter-photon:v2.0.0

docker push registry.test.paas/library/chartmuseum-photon:v2.0.0

vi /opt/harbor/docker-compose.yml

=> tag image 편집

------------------------------------------------------------

-- 4. Download the Harbor Installer

------------------------------------------------------------

* Online installer: The online installer downloads the Harbor images from Docker hub. For this reason, the installer is very small in size.

* Offline installer: Use the offline installer if the host to which are are deploying Harbor does not have a connection to the Internet.

The offline installer contains pre-built images, so it is larger than the online installer.

● Download and Unpack the Installer

https://github.com/goharbor/harbor/releases

#wget https://github.com/goharbor/harbor/releases/download/v2.0.0/harbor-offline-installer-v2.0.0.tgz

wget https://storage.googleapis.com/harbor-releases/release-2.0.0/harbor-online-installer-v2.0.0.tgz

tar xvf harbor-online-installer-v2.0.0.tgz

mv /app/harbor /opt

ls -l /opt

root docker 94 Jun 2 10:58 harbor

------------------------------------------------------------

-- 5. Registry disk setting (optional)

------------------------------------------------------------

● partitioning disk

------------------------------------------------------------

fdisk -l

fdisk /dev/sdb

Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.

Be careful before using the write command.

Device does not contain a recognized partition table

Building a new DOS disklabel with disk identifier 0xf3f4d873.

Command (m for help): n

Partition type:

p primary (0 primary, 0 extended, 4 free)

e extended

Select (default p): p

Partition number (1-4, default 1):

First sector (2048-209715199, default 2048):

Using default value 2048

Last sector, +sectors or +size{K,M,G} (2048-209715199, default 209715199):

Using default value 209715199

Partition 1 of type Linux and of size 100 GiB is set

Command (m for help): p

Disk /dev/sdc: 107.4 GB, 107374182400 bytes, 209715200 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0xf3f4d873

Device Boot Start End Blocks Id System

/dev/sdc1 2048 209715199 104856576 83 Linux

Command (m for help): t

Selected partition 1

Hex code (type L to list all codes): 8e

Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): w

The partition table has been altered!

yum install lvm2

fdisk -l

pvcreate /dev/sdb1

vgcreate registry-vg /dev/sdb1

lvcreate -n registry-lv -l 100%FREE registry-vg

#mkfs.ext4 /dev/mapper/registry--vg-registry--lv

mkfs.xfs /dev/mapper/registry--vg-registry--lv

mkfs.xfs -f -ssize=4k /dev/mapper/registry--vg-registry--lv

fsck -y /dev/mapper/registry--vg-registry--lv

mkdir /harbor-data

mount /dev/mapper/registry--vg-registry--lv /harbor-data

vi /etc/fstab

#/dev/mapper/registry--vg-registry--lv /harbor-data ext4 defaults 0 0

/dev/mapper/registry--vg-registry--lv /harbor-data xfs defaults 0 0

● install NFS Server

------------------------------------------------------------

yum install nfs-utils libnfsidmap

systemctl enable rpcbind

systemctl enable nfs-server

systemctl start rpcbind

systemctl start nfs-server

systemctl start rpc-statd

systemctl start nfs-idmapd

systemctl enable nfs.service

systemctl start nfs.service

chkconfig nfs on

mkdir -p /harbor-data/registry

chmod 750 /harbor-data/registry

chown nfsnobody:nfsnobody /harbor-data/registry

mkdir -p /harbor-data/db-data

chmod 750 /harbor-data/db-data

chown nfsnobody:nfsnobody /harbor-data/db-data

vi /etc/exports

/harbor-data/registry *(rw,async,all_squash)

/harbor-data/db-data *(rw,async,all_squash)

exportfs -a

setsebool -P virt_use_nfs on (server, client 둘다 등록)

● configure NFS server firewall

-------------------------------------------------------------

# add-service

firewall-cmd --permanent --zone public --add-service mountd

firewall-cmd --permanent --zone public --add-service rpc-bind

firewall-cmd --permanent --zone public --add-service nfs

firewall-cmd --reload

# add-port(server, client 둘다 등록)

firewall-cmd --permanent --zone=public --add-port=53248/tcp

firewall-cmd --permanent --zone=public --add-port=50825/tcp

firewall-cmd --permanent --zone=public --add-port=20048/tcp

firewall-cmd --permanent --zone=public --add-port=2049/tcp

firewall-cmd --permanent --zone=public --add-port=111/tcp

firewall-cmd --reload

exportfs -r

exportfs -v

client)

mount -t nfs registry.test.paas:/harbor-data /mnt

------------------------------------------------------------

-- 6. Configure HTTPS Access to Harbor

------------------------------------------------------------

mkdir -p ./openssl

cd openssl

● Generate a Certificate Authority Certificate

1) Generate a CA certificate private key

openssl genrsa -out ca.key 4096

2) Generate the CA certificate

openssl req -x509 -new -nodes -sha512 -days 3650 \

-subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=registry.test.paas" \

-key ca.key \

-out ca.crt

● Generate a Server Certificate

1) Generate a private key

openssl genrsa -out registry.test.paas.key 4096

2) Generate a certificate signing request (CSR)

openssl req -sha512 -new \

-subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=registry.test.paas" \

-key registry.test.paas.key \

-out registry.test.paas.csr

3) Generate an x509 v3 extension file

cat > v3.ext <<-EOF

authorityKeyIdentifier=keyid,issuer

basicConstraints=CA:FALSE

keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment

extendedKeyUsage = serverAuth

subjectAltName = @alt_names

[alt_names]

DNS.1=registry.test.paas

DNS.2=registry

DNS.3=harbor.test.paas

EOF

4) Use the v3.ext file to generate a certificate for your Harbor host

openssl x509 -req -sha512 -days 3650 \

-extfile v3.ext \

-CA ca.crt -CAkey ca.key -CAcreateserial \

-in registry.test.paas.csr \

-out registry.test.paas.crt

● Provide the Certificates to Harbor and Docker

1) Copy the server certificate and key into the certficates folder on your Harbor host.

mkdir -p /harbor-data/cert/

nfs)

chmod 750 /harbor-data/cert

chown nfsnobody:nfsnobody /harbor-data/cert

vi /etc/exports

/harbor-data/cert *(rw,async,all_squash)

exportfs -a

cp registry.test.paas.crt /harbor-data/cert/

cp registry.test.paas.key /harbor-data/cert/

2) Convert registry.test.paas.crt to registry.test.paas.cert, for use by Docker.

openssl x509 -inform PEM -in registry.test.paas.crt -out registry.test.paas.cert

# openssl x509 -inform PEM -in ca.crt -out ca.cert

3) Copy the server certificate, key and CA files into the Docker certificates folder on the Harbor host. You must create the appropriate folders first.

# http (80)

mkdir -p /etc/docker/certs.d/registry.test.paas/

cp registry.test.paas.cert /etc/docker/certs.d/registry.test.paas/

cp registry.test.paas.key /etc/docker/certs.d/registry.test.paas/

cp ca.crt /etc/docker/certs.d/registry.test.paas/

# https (443)

mkdir -p /etc/docker/certs.d/registry.test.paas:443/

cp registry.test.paas.cert /etc/docker/certs.d/registry.test.paas:443/

cp registry.test.paas.key /etc/docker/certs.d/registry.test.paas:443/

cp ca.crt /etc/docker/certs.d/registry.test.paas:443/

cp registry.test.paas.crt /etc/pki/ca-trust/source/anchors/registry.test.paas.crt

update-ca-trust

# Restart Docker Engine.

systemctl restart docker

systemctl status docker

/etc/docker/certs.d/

└── yourdomain.com:port

├── yourdomain.com.cert <-- Server certificate signed by CA

├── yourdomain.com.key <-- Server key signed by CA

└── ca.crt <-- Certificate authority that signed the registry certificate

------------------------------------------------------------

-- 7. Configure Internal TLS communication between Harbor Component

------------------------------------------------------------

mkdir -p /harbor-data/tls/cert

nfs)

chmod 750 /harbor-data/tls

chown nfsnobody:nfsnobody /harbor-data/tls

vi /etc/exports

/harbor-data/tls *(rw,async,all_squash)

exportfs -a

cp /harbor-install/openssl/cat.crt /harbor-data/tls/cert/

cp /harbor-install/openssl/cat.key /harbor-data/tls/cert/

cp /harbor-install/openssl/v3.ext /harbor-data/tls/cert/

cd /harbor-data/tls/cert

1) harbor_internal_ca

------------------------------------------------------------

openssl genrsa -out harbor_internal_ca.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=harbor_internal_ca" -key harbor_internal_ca.key -out harbor_internal_ca.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in harbor_internal_ca.csr -out harbor_internal_ca.crt

2) core

------------------------------------------------------------

openssl genrsa -out core.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=core" -key core.key -out core.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in core.csr -out core.crt

3) job_service

------------------------------------------------------------

openssl genrsa -out job_service.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=job_service" -key job_service.key -out job_service.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in job_service.csr -out job_service.crt

4) proxy

------------------------------------------------------------

openssl genrsa -out proxy.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=proxy" -key proxy.key -out proxy.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in proxy.csr -out proxy.crt

5) portal

------------------------------------------------------------

openssl genrsa -out portal.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=portal" -key portal.key -out portal.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in portal.csr -out portal.crt

6) registry

------------------------------------------------------------

openssl genrsa -out registry.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=registry" -key registry.key -out registry.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in registry.csr -out registry.crt

7) registryctl

------------------------------------------------------------

openssl genrsa -out registryctl.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=registryctl" -key registryctl.key -out registryctl.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in registryctl.csr -out registryctl.crt

8) notary_server

------------------------------------------------------------

openssl genrsa -out notary_server.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=notary_server" -key notary_server.key -out notary_server.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in notary_server.csr -out notary_server.crt

9) notary_signer

------------------------------------------------------------

openssl genrsa -out notary_signer.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=notary_signer" -key notary_signer.key -out notary_signer.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in notary_signer.csr -out notary_signer.crt

10) trivy_adapter

------------------------------------------------------------

openssl genrsa -out trivy_adapter.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=trivy_adapter" -key trivy_adapter.key -out trivy_adapter.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in trivy_adapter.csr -out trivy_adapter.crt

11) clair

------------------------------------------------------------

openssl genrsa -out clair.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=clair" -key clair.key -out clair.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in clair.csr -out clair.crt

12) clair_adapter

------------------------------------------------------------

openssl genrsa -out clair_adapter.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=clair_adapter" -key clair_adapter.key -out clair_adapter.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in clair_adapter.csr -out clair_adapter.crt

13) chartmuseum

------------------------------------------------------------

openssl genrsa -out chartmuseum.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=freedream/OU=IT Department/CN=chartmuseum" -key chartmuseum.key -out chartmuseum.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in chartmuseum.csr -out chartmuseum.crt

# offline docker images save

docker save -o goharbor.tar goharbor/prepare:v2.0.0

docker run -v /:/harbor-data goharbor/prepare:v2.0.0 gencert -p /harbor-data/tls/cert

------------------------------------------------------------

-- 8. Configure the Harbor YML File

------------------------------------------------------------

● Configure the Harbor YML File

cd harbor

cp harbor.yml.tmpl harbor.yml

# vi harbor.yml

------------------------------------------------------------

hostname: registry.test.paas

# http related config

http:

# port for http, default is 80. If https enabled, this port will redirect to https port

port: 80

https:

# https port for harbor, default is 443

port: 443

# The path of cert and key files for nginx

certificate: /harbor-data/cert/registry.test.paas.crt

private_key: /harbor-data/cert/registry.test.paas.key

# enable tls communication between all harbor components

internal_tls:

# # set enabled to true means internal tls is enabled

enabled: true

# # put your cert and key files on dir

dir: /harbor-data/tls/cert

harbor_admin_password: Harbor12345

# Harbor DB configuration

database:

# The password for the root user of Harbor DB. Change this before any production use.

password: root123

# The maximum number of connections in the idle connection pool. If it <=0, no idle connections are retained.

max_idle_conns: 50

# The maximum number of open connections to the database. If it <= 0, then there is no limit on the number of open connections.

# Note: the default number of connections is 100 for postgres.

max_open_conns: 100

# The default data volume

data_volume: /harbor-data/db-data

# Harbor Storage settings by default is using /data dir on local filesystem

# Uncomment storage_service setting If you want to using external storage

# storage_service:

# # ca_bundle is the path to the custom root ca certificate, which will be injected into the truststore

# # of registry's and chart repository's containers. This is usually needed when the user hosts a internal storage with self signed certificate.

# ca_bundle:

# # storage backend, default is filesystem, options include filesystem, azure, gcs, s3, swift and oss

# # for more info about this configuration please refer https://docs.docker.com/registry/configuration/

# filesystem:

# maxthreads: 100

# # set disable to true when you want to disable registry redirect

# redirect:

# disabled: false

...

# Log configurations

log:

# options are debug, info, warning, error, fatal

level: info

# configs for logs in local storage

local:

# Log files are rotated log_rotate_count times before being removed. If count is 0, old versions are removed rather than rotated.

rotate_count: 50

# Log files are rotated only if they grow bigger than log_rotate_size bytes. If size is followed by k, the size is assumed to be in kilobytes.

# If the M is used, the size is in megabytes, and if G is used, the size is in gigabytes. So size 100, size 100k, size 100M and size 100G

# are all valid.

rotate_size: 200M

# The directory on your host that store log

location: /var/log/harbor

------------------------------------------------------------

cd /opt/openssl

openssl genrsa -out harbor_db.key 4096

openssl req -sha512 -new -subj "/C=KR/ST=Seoul/L=Seoul/O=INSoft/OU=Cloud Department/CN=harbor_db" -key harbor_db.key -out harbor_db.csr

openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in harbor_db.csr -out harbor_db.crt

mkdir -p /harbor-data/db-data/secret/tls

cp harbor_db.crt /harbor-data/db-data/secret/tls/

cp harbor_db.key /harbor-data/db-data/secret/tls/

cp harbor_db.csr /harbor-data/db-data/secret/tls/

chown -R 10000:10000 /harbor-data/db-data/secret/tls/

tail -f /var/log/harbor/registry.log

------------------------------------------------------------

-- 9. Run the Installer Script

------------------------------------------------------------

1) Default installation without Notary, Clair, or Chart Repository Service

cd /opt/harbor

# Connecting to Harbor via HTTP

vi /etc/docker/daemon.json

{

"insecure-registries" : ["myregistrydomain.com:5000", "0.0.0.0"]

}

./install.sh

https://registry.test.paas admin / Harbor12345

docker login reg.yourdomain.com

docker push reg.yourdomain.com/myproject/myrepo:mytag

2) Installation with Notary, Clair, and Chart Repository Service

cd /opt/harbor

# Run the prepare script to enable HTTPS

./prepare --with-notary --with-clair --with-chartmuseum

./install.sh --with-notary --with-clair --with-chartmuseum

# If Harbor is running, stop and remove the existing instance

docker-compose down -v

3) Restart Harbor

docker-compose stop

docker-compose start

docker-compose restart

4) Reconfigure Harbor

# stop Harbor

docker-compose down -v

# update

vim harbor.yml

# populate the configuration

./prepare

# start Harbor

docker-compose up -d

docker-compose ps

# remove docker-compose

docker-compose rm -f

5) Verify the HTTPS Connection

https://registry.test.paas admin / Harbor12345

systemctl restart docker

systemctl status docker

firewall-cmd --permanent --zone=public --add-port=443/tcp

firewall-cmd --reload

6) Harbor management

# harbor console

https://registry.test.paas

admin / Harbor12345

# create harbor user

Harbor console > Administration > User > New User

set as admin 권한 부여

# docker login to registry

docker login registry.test.paas

harbor / Harbor12345

# docker image push to registry

docker pull nginx:latest

docker tag nginx:latest registry.test.paas/library/nginx:latest

docker push registry.test.paas/library/nginx:latest

# docker image pull from registry

# docker rmi nginx:latest

# docker rmi registry.test.paas/library/nginx:latest

docker pull registry.test.paas/library/nginx:latest

Kubernetes Off-Line Install Guide

freedream — Wed, 19 Aug 2020 17:24:26 +0900

Kubernetes Off-Line Install Guide

● OS: CentOS8 x86_64 minimal

● docker: 18.09.1

● kubernetes: 1.17

1. Yum Repository Server 구성

1) Online Obtaining Repository Packages

a) Install required packages

sudo yum -y install yum-utils createrepo git

b) Make a directory to store the software in the server’s storage

mkdir -p /var/www/html/repos

● SELinux 비활성화

setenforce 0

sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config

# permissive mode

sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

c) Repository 추가

● docker ce repository 추가

sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo

centos7) yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

[docker-ce-stable]

name=Docker CE Stable - $basearch

baseurl=https://download.docker.com/linux/centos/7/$basearch/stable

#baseurl=https://download.docker.com/linux/centos/7/x86_64/stable

enabled=1

gpgcheck=1

gpgkey=https://download.docker.com/linux/centos/gpg

yum install docker-ce -y

● kubernetes repository 추가

cat <<EOF > /etc/yum.repos.d/kubernetes.repo

[kubernetes]

name=Kubernetes

baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64

enabled=1

gpgcheck=1

repo_gpgcheck=1

gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

EOF

● CentOS8 BaseOS 추가

cat <<EOF > /etc/yum.repos.d/CentOS-Base.repo

[centos8-base]

name=CentOS-$releasever - Base

mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=BaseOS&infra=$infra

#baseurl=http://mirror.centos.org/$contentdir/$releasever/BaseOS/$basearch/os/

#baseurl=http://mirror.centos.org/centos/8.1.1911/BaseOS/x86_64/os/

gpgcheck=1

enabled=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial

EOF

● CentOS8 Extras 추가

cat <<EOF > /etc/yum.repos.d/CentOS-Extras.repo

[centos8-extras]

name=CentOS-$releasever - Extras

mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra

#baseurl=http://mirror.centos.org/$contentdir/$releasever/extras/$basearch/os/

gpgcheck=1

enabled=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial

EOF

● CentOS8 AppStream 추가

cat <<EOF > /etc/yum.repos.d/CentOS-AppStream.repo

[centos8-appstream]

name=CentOS-$releasever - AppStream

mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=AppStream&infra=$infra

#baseurl=http://mirror.centos.org/$contentdir/$releasever/AppStream/$basearch/os/

gpgcheck=1

enabled=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial

EOF

d) Sync the packages and create the repository

dnf update; dnf repolist;

for repo in \

centos8-base \

centos8-extras \

centos8-appstream \

docker-ce-stable \

kubernetes

reposync --gpgcheck -lm --repoid=${repo} --download_path=/var/www/html/repos

createrepo -v /var/www/html/repos/${repo} -o /var/www/html/repos/${repo}

done

e) Compress tar Repository Packages

tar -cvf kubenetes-repo.tar /var/www/html/repos

2) Offline Install Repository Server

a) Prepare the webserver

dnf install httpd

b) Extract tar Repository Packages

tar -xvzf kubenetes-repo.tar /var/www/html/

ls /var/www/html/repos

c) Permission repository files

chmod -R +r /var/www/html/repos

restorecon -vR /var/www/html

d) Add the firewall rules

sudo firewall-cmd --permanent --add-service=http

sudo firewall-cmd --reload

e) Enable and start webserver

systemctl enable httpd

systemctl start httpd

2. Kubernetes Offline Installation

1) Prepare Packages/Images for Kubernetes

a) Obtaining Packages

※ rpm package로 설치 시 (offline yum server 구성 시 생략)

yum makecache --timer

yumdownloader --resolve kubelet kubeadm kubectl

35625b6ab1da6c58ce4946742181c0dcf9ac9b6c2b5bea2c13eed4876024c342-kubectl-1.17.3-0.x86_64.rpm

3f1db71d0bb6d72bc956d788ffee737714e5717c629b26355a2dcf1dba4ad231-kubelet-1.17.3-0.x86_64.rpm

d0c889d5fae925a9266aa521f6aed361ea53792c86a51c38f5d92cd3f03e2d30-kubeadm-1.17.3-0.x86_64.rpm

yumdownloader --assumeyes --destdir=<your_rpm_dir> --resolve yum-utils kubeadm-1.17.* kubelet-1.17.* kubectl-1.17.* ebtables

yum install -y --cacheonly --disablerepo=* <your_rpm_dir>/*.rpm

kubeadm config images list

b) Obtaining images

● pull kubernetes images

docker pull k8s.gcr.io/kube-apiserver:v1.17.3

docker pull k8s.gcr.io/kube-controller-manager:v1.17.3

docker pull k8s.gcr.io/kube-scheduler:v1.17.3

docker pull k8s.gcr.io/kube-proxy:v1.17.3

docker pull k8s.gcr.io/pause:3.1

docker pull k8s.gcr.io/etcd:3.4.3-0

docker pull k8s.gcr.io/coredns:1.6.5

docker pull kubernetesui/dashboard:v2.0.3

docker pull kubernetesui/metrics-scraper:v1.0.4

docker pull weaveworks/weave-npc:2.6.0

docker pull weaveworks/weave-kube:2.6.0

docker pull quay.io/coreos/flannel:v0.11.0-amd64

※ dashboard

kubernetesui/dashboard:v2.0.3

kubernetesui/metrics-scraper:v1.0.4

https://github.com/kubernetes/dashboard/releases

=> kubernetes 지원 버전에 맞는 dashboard 버전 설치

Kubernetes version	1.15	1.16	1.17	1.18
Compatibility	?	?	?	✓

● Exporting images

# Create a directory to store

mkdir -p ~/k8s-images

cd ~/k8s-images

# Export kubernetes images

docker save -o kube-master-images.tar \

k8s.gcr.io/kube-proxy:v1.17.3 \

k8s.gcr.io/kube-apiserver:v1.17.3 \

k8s.gcr.io/kube-controller-manager:v1.17.3 \

k8s.gcr.io/kube-scheduler:v1.17.3 \

k8s.gcr.io/coredns:1.6.5 \

k8s.gcr.io/etcd:3.4.3-0 \

k8s.gcr.io/pause:3.1

docker save -o kube-node-images.tar \

k8s.gcr.io/kube-proxy:v1.17.3 \

k8s.gcr.io/coredns:1.6.5 \

k8s.gcr.io/pause:3.1 \

jettech/kube-webhook-certgen:v1.0.0

docker save -o dashboard-images.tar \

kubernetesui/dashboard:v2.0.0-rc5 \

kubernetesui/metrics-scraper:v1.0.3

docker save -o network-images.tar \

weaveworks/weave-npc:2.6.0 \

weaveworks/weave-kube:2.6.0 \

quay.io/coreos/flannel:v0.11.0-amd64

c) pod-network download

● Weave network download

export kubever=$(kubectl version | base64 | tr -d '\n')

wget https://cloud.weave.works/k8s/net?k8s-version=$kubever

● kube-flannel download

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

d) dashboard download

https://github.com/kubernetes/dashboard/releases

=> kubernetes 지원 버전에 맞는 dashboard 버전 설치

wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-rc5/aio/deploy/recommended.yaml

2) VM 환경 구성

● Architecture

hostname address cpu memory disk

------------------------------------------------------------------------

master.k8s.paas x.x.20.11 4 8G 300G

node01.k8s.paas x.x.20.21 2 4G 300G

node02.k8s.paas x.x.20.22 2 4G 300G

● hostname 변경

hostnamectl --static set-hostname master01.k8s.paas

hostnamectl

hostname

vi /etc/hosts

x.x.20.11 master.k8s.paas

x.x.20.21 node01.k8s.paas

x.x.20.22 node02.k8s.paas

● network connection 설정

# create a connection with the name enp0s8

# nmcli con add type ethernet con-name enp0s3 ifname enp0s3

nmcli con mod enp0s3 ipv4.address x.x.20.11/24

nmcli con mod enp0s3 ipv4.gateway x.x.20.1

nmcli con mod enp0s3 ipv4.method manual

nmcli con mod enp0s3 +ipv4.dns 8.8.8.8

nmcli con mod enp0s3 +ipv4.dns-search k8s.paas

nmcli con mod enp0s3 connection.autoconnect yes

nmcli con up enp0s3

nmcli con show enp0s3

more /etc/resolv.conf

more /etc/sysconfig/network-scripts/ifcfg-enp0s3

nmcli dev show enp0s3

nmcli dev status

#nmcli con del enp0s3

#nmcli con del f653ccb6-4a12-496b-bceb-78f0ea33aa72

ip a

ip r

● 방화벽 설정

# systemctl disable firewalld && systemctl stop firewalld

▶ Kubernetes service ports in Linux firewall

# Master node

----------------------------------------------------------------

firewall-cmd --add-port={6443,2379-2380,10250,10251,10252,5473,179,5473}/tcp --permanent

firewall-cmd --add-port={4789,8285,8472}/udp --permanent

firewall-cmd --reload

# Worker nodes

----------------------------------------------------------------

firewall-cmd --add-port={10250,30000-32767,5473,179,5473}/tcp --permanent

firewall-cmd --add-port={4789,8285,8472}/udp --permanent

firewall-cmd --reload

● 필요한 패키지 설치

yum install yum-utils device-mapper-persistent-data lvm2 -y

yum install wget git net-tools bind-utils iptables-services bash-completion kexec-tools sos psacct vim -y

b) Yum Repository 추가 (매 vm 별로 추가)

cat <<EOF > /etc/yum.repos.d/kubernetes-offline.repo

[docker-ce-stable]

name=docker-ce-stable

baseurl=http://<server_IP>/repos/docker-ce-stable

enabled=1

gpgcheck=0

[kubernetes]

name=kubernetes

baseurl=http://<server_IP>/repos/kubernetes

enabled=1

gpgcheck=0

[centos8-base]

name=centos8-base

baseurl=http://<server_IP>/repos/centos8-base

enabled=1

gpgcheck=0

[centos8-extras]

name=centos8-extras

baseurl=http://<server_IP>/repos/centos8-extras

enabled=1

gpgcheck=0

[centos8-appstream]

name=centos8-appstream

baseurl=http://<server_IP>/repos/centos8-appstream

enabled=1

gpgcheck=0

EOF

c) install Docker

● docker version list

dnf list docker-ce --showduplicates | sort -r

docker-ce.x86_64 3:18.09.1-3.el7 @docker-ce-stable <- 안정된 버전이 @로 표시됨

● Install docker

# latest version

dnf install --nobest docker-ce

dnf install docker-ce

# stable version

dnf update && dnf install docker-ce-3:18.09.1-3.el7 --allowerasing

systemctl enable --now docker

newgrp docker

docker version

usermod -aG docker $USER

usermod -aG docker paasuser

# remove docker group

#gpasswd -d paasuser docker

● daemon.json 수정

cat > /etc/docker/daemon.json <<EOF

{

"exec-opts": ["native.cgroupdriver=systemd"],

"log-driver": "json-file",

"log-opts": {

"max-size": "100m"

"storage-driver": "overlay2",

"storage-opts": [

"overlay2.override_kernel_check=true"

"insecure-registries": ["registry.k8s.paas:5000","172.30.0.0/16"]

}

EOF

● docker 재시작

systemctl daemon-reload

systemctl restart docker

systemctl status docker

d) Install Kubernetes packages

● kubernetes package 설치

yum update && systemctl reboot

yum install -y epel-release kubelet kubeadm kubectl kubernetes-cni --disableexcludes=kubernetes

systemctl enable --now kubelet

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.19.0-rc.4/bin/linux/amd64/kubectl

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.17.9/bin/linux/amd64/kubectl

https://github.com/kubernetes/kubernetes/releases

yumdownloader --assumeyes --destdir=<your_rpm_dir> --resolve yum-utils kubeadm-1.17.* kubelet-1.17.* kubectl-1.17.* ebtables

yum install -y --cacheonly --disablerepo=* <your_rpm_dir>/*.rpm

kubeadm config images list

※ rpm package로 설치 시

rpm -ivh --replacefiles --replacepkgs ~/k8s-images/*.rpm

● net bridge 등록

cat <<EOF > /etc/sysctl.d/k8s.conf

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

EOF

modprobe br_netfilter

modprobe overlay

echo '1' > /proc/sys/net/ipv4/ip_forward

sysctl --system

● swap 비활성화

swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab

cat /etc/fstab

#/dev/mapper/cl-swap swap swap defaults 0 0

● Configure cgroup driver used by kubelet on control-plane node

vi /var/lib/kubelet/config.yaml

apiVersion: kubelet.config.k8s.io/v1beta1

kind: KubeletConfiguration

cgroupDriver: <value>

e) VM 복제

※ master서버를 구성 후 이를 node서버로 복제하여 사용할 경우

● VM의 hostname 변경

hostnamectl --static set-hostname node01.k8s.paas

hostnamectl

hostname

vi /etc/hosts

x.x.20.11 master.k8s.paas

x.x.20.21 node01.k8s.paas

x.x.20.22 node02.k8s.paas

● network connection 설정

nmcli con mod enp0s3 ipv4.address x.x.25.61/24

nmcli con up enp0s3

nmcli con show enp0s3

more /etc/resolv.conf

more /etc/sysconfig/network-scripts/ifcfg-enp0s3

● ssh keygen

ssh-keygen

cat ~/.ssh/id_rsa.pub

ssh-copy-id master01

ssh-copy-id worker01

ssh-copy-id worker02

cat >> ~/.ssh/authorized_keys <<EOF

ssh-rsa ...root@master01.k8s.paas

EOF

ssh master01

ssh worker01

ssh worker02

cat ~/.ssh/authorized_keys

● 각 VM별 Kubernetes images Load

# master

docker load -i ~/k8s-images/kube-master-images.tar

# node

docker load -i ~/k8s-images/kube-node-images.tar

# network (master, node)

docker load -i ~/k8s-images/kube-network-images.tar

# dashboard (master, node)

docker load -i ~/k8s-images/kube-dashboard-images.tar

3) Offline Install Kubernetes

a) Configure a master node

# centos8)

dnf provides tc

dnf install iproute-tc

# Initialize your control-plane node

lsmod | grep br_netfilter

systemctl enable kubelet

kubeadm config images pull

kubeadm init --pod-network-cidr=10.244.0.0/16 --kubernetes-version=v1.18.6

kubeadm init

#kubeadm init --ignore-preflight-errors=all --pod-network-cidr 10.244.0.0/16

kubeadm alpha certs certificate-key

cd35c5f3a27bf418bbc9f1f778e9f7fe865f008dc404a2b2c44dbe2a4f314f12

kubeadm init phase upload-certs --upload-certs --certificate-key=SOME_VALUE --ignore-preflight-errors=all --pod-network-cidr 10.244.0.0/16

kubeadm init phase bootstrap-token

https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.

Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.x.x:6443 --token qza4mi.qtcm7soz32jwatpi \

--discovery-token-ca-cert-hash sha256:553f30ff39cbf3929cd30c8e2d57109641a43a21e6ec4a909e512512f9789756

b) KUBECONFIG 설정

unset KUBECONFIG

echo 'export KUBECONFIG=/etc/kubernetes/admin.conf' >> ~/.bashrc

● Enable bash completion for kubectl

yum install bash-completion -y

kubectl completion bash >/etc/bash_completion.d/kubectl

echo 'alias k=kubectl' >> ~/.bashrc

echo 'complete -F __start_kubectl k' >> ~/.bashrc

source ~/.bashrc

c) cluster 사용 설정 (master node)

mkdir -p $HOME/.kube

cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

chown $(id -u):$(id -g) $HOME/.kube/config

● client서버에서 cluster 사용 시

scp root@master01.k8s.paas:/etc/kubernetes/admin.conf ~/.kube/config

chown $(id -u):$(id -g) ~/.kube/config

chmod 644 /root/.kube/config

echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc

source ~/.bashrc

● Start and enable kubelet.service

systemctl enable kubelet.service

systemctl start kubelet.service

systemctl status kubelet.service

d) pod network 설정 (master)

# Flannel network

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

kubectl apply -f ~/k8s-images/kube-flannel.yml

kubectl get pods -o wide -w -n kube-system

kubectl -n kube-system get pods -l app=flannel

# Weave network

export kubever=$(kubectl version | base64 | tr -d '\n')

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$kubever"

kubectl get pods --all-namespaces

kubectl apply -f ~/k8s-images/weave.yml

sudo curl -L git.io/weave -o /usr/local/bin/weave

sudo chmod a+x /usr/local/bin/weave

# calico 사용 시 (192.168.0.0/16)

kubeadm init --pod-network-cidr=192.168.0.0/16 --apiserver-advertise-address={ip address}

kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml

kubectl taint nodes --all node-role.kubernetes.io/master-

#systemctl restart docker

#systemctl status docker

e) join worker nodes

※ worker node에서 실행)

kubeadm join $cluster-ip:6443 --token 9rfr3o.n1i8di8exqeut723 \

--discovery-token-ca-cert-hash sha256:4184bd55ac8a8daf29049fd68fe24a4e258fef0b1e2289a5e77b66adc57b2084

kubectl get pods -n kube-system

kubectl get nodes

kubectl get po --all-namespaces

f) kubeadm re-install

# Master node

kubectl drain master.k8s.paas --delete-local-data --force --ignore-daemonsets

iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X

#ipvsadm -C

kubectl delete node master.k8s.paas

kubeadm reset

docker rm -f $(docker ps -a -q)

rm -f $HOME/.kube/config

rm -rf /etc/kubernetes

rm -rf /var/lib/kubelet

rm -rf /var/lib/etcd

rm -rf /etc/cni/net.d

# Worker node

kubeadm reset

docker rm -f $(docker ps -a -q)

#kubeadm init phase bootstrap-token

3. Dashboard Installation (Off-line)

1) Install Dashboard

# kubectl delete ns kubernetes-dashboard

a) cluster-admin ClusterRole Binding 적용

vim ~/k8s-images/recommended.yaml

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

roleRef:

apiGroup: rbac.authorization.k8s.io

kind: ClusterRole

subjects:

- kind: ServiceAccount

namespace: kubernetes-dashboard

b) install dashboard

kubectl apply -f ~/k8s-images/recommended.yaml

2) Dashboard login

a) API Server를 활용하는 방법

grep 'client-certificate-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.crt

cat kubecfg.crt

grep 'client-key-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.key

cat kubecfg.key

openssl pkcs12 -export -clcerts -inkey kubecfg.key -in kubecfg.crt -out kubecfg.p12 -name "kubernetes-ha"

# 신뢰할 수 있는 루트인증기관 인증서 적용

certutil.exe -addstore "Root" D:\app\cert\ca.crt

# 개인용 인증서 적용

certutil.exe -p root -user -importPFX D:\app\cert\kubecfg.p12

# 인증서 확인/변경

certmgr.msc

b) dashboard login

kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep kubernetes-dashboard | awk '{print $1}')

APISERVER=$(kubectl config view --minify | grep server | cut -f 2- -d ":" | tr -d " ")

SECRET_NAME=$(kubectl -n kubernetes-dashboard get secrets | grep ^kubernetes-dashboard-token | cut -f1 -d ' ')

TOKEN=$(kubectl -n kubernetes-dashboard describe secret $SECRET_NAME | grep -E '^token' | cut -f2 -d':' | tr -d " ")

echo $TOKEN

curl $APISERVER/api --header "Authorization: Bearer $TOKEN" --insecure

https://:$IP:6443/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/

https://phoenixnap.com/kb/how-to-install-kubernetes-on-centos

https://computingforgeeks.com/install-kubernetes-cluster-on-centos-with-kubeadm/

https://docs.openshift.com/container-platform/3.11/install/disconnected_install.html

https://ahmermansoor.blogspot.com/2019/04/install-kubernetes-k8s-offline-on-centos-7.html

https://kubernetes.io/ko/docs/tasks/access-application-cluster/web-ui-dashboard/

Using JBoss 7.2 Image and Template for OpenShift

freedream — Mon, 29 Jul 2019 17:51:17 +0900

1. On line Imagestreams and Templates Import
--------------------------------------------------------------------------------
for resource in eap72-image-stream.json \
  eap72-amq-persistent-s2i.json \
  eap72-amq-s2i.json \
  eap72-basic-s2i.json \
  eap72-https-s2i.json \
  eap72-mongodb-persistent-s2i.json \
  eap72-mongodb-s2i.json \
  eap72-mysql-persistent-s2i.json \
  eap72-mysql-s2i.json \
  eap72-postgresql-persistent-s2i.json \
  eap72-postgresql-s2i.json \
  eap72-sso-s2i.json \
  eap72-third-party-db-s2i.json \
  eap72-tx-recovery-s2i.json
do
  oc replace -n openshift --force -f https://raw.githubusercontent.com/jboss-container-images/jboss-eap-7-openshift-image/eap72/templates/${resource}
done

2. Off Line Imagestreams and Templates Import
--------------------------------------------------------------------------------
2.1 Obtaining images (on line)
docker login registry.redhat.io
docker pull registry.redhat.io/jboss-eap-7/eap72-openshift:latest
docker save -o eap72-openshift.tar registry.redhat.io/jboss-eap-7/eap72-openshift:latest

2.2 private docker registry Image Import (off line)
※ prerequisites: private docker registry (distribution docker, Quay, etc)

docker load -i eap72-openshift.tar

docker login -u user -p $(oc whoami -t) registry.example.com:5000
docker tag registry.redhat.io/jboss-eap-7/eap72-openshift:latest registry.example.com:5000/jboss-eap-7/jboss-eap72-openshift:latest
docker tag registry.redhat.io/jboss-eap-7/eap72-openshift:latest registry.example.com:5000/jboss-eap-7/jboss-eap72-openshift:1.0
docker push registry.example.com:5000/jboss-eap-7/jboss-eap72-openshift:latest
docker push registry.example.com:5000/jboss-eap-7/jboss-eap72-openshift:1.0

2.3 Obtaining and Modifing ImageStream template (ex. eap72-image-stream.json)
git clone https://github.com/jboss-container-images/jboss-eap-7-openshift-image.git
cd jboss-eap-7-openshift-image
vi eap72-image-stream.json
# DockerImage
registry.redhat.io/jboss-eap-7/eap72-openshift:latest => registry.example.com:5000/jboss-eap-7/eap72-openshift:latest
registry.redhat.io/jboss-eap-7/eap72-openshift:1.0 => registry.example.com:5000/jboss-eap-7/eap72-openshift:1.0

2.4 Importing ImageStreams and Templates
oc create -f templates -n openshift

※ push images from the integrated container registry directly
docker login -p $(oc whoami -t) -e unused -u unused docker-registry-default.apps.example.com
docker tag registry.example.com:5000/jboss-eap-7/jboss-eap72-openshift:latest docker-registry-default.apps.example.com/openshift/jboss-eap72-openshift:latest
docker push docker-registry-default.apps.example.com/openshift/jboss-eap72-openshift:latest

2.5 Updating and Tagging Existing Images
oc import-image registry.example.com:5000/jboss-eap-7/jboss-eap72-openshift:latest --insecure=true --confirm -n openshift
oc tag jboss-eap72-openshift:latest jboss-eap72-openshift:1.0 -n openshift

3. Template spec check (parameter)
--------------------------------------------------------------------------------
oc process eap72-mysql-s2i --parameters -n openshift > tem_eap72-mysql-s2i_spec.txt

4. Creating New Applications With Templates
--------------------------------------------------------------------------------
● Application Building with parameter settings

# build.sh

#!/bin/sh

PROJECT_NAME=eap72-mysql-s2i
SVC_ACCOUNT_NAME=eap7-service-account
SECRET_NAME=eap7-app-secret
CUSTOM_DB_USERNAME=openshift
CUSTOM_DB_PASSWORD=password
CUSTOM_DB_NAME=mysql57
TEMPLATE_NAME=eap72-mysql-s2i

oc new-project $PROJECT_NAME --display-name="eap72-mysql-s2i" --description="EAP72-MYSQL-S2I TEST"
oc project $PROJECT_NAME
oc create serviceaccount $SVC_ACCOUNT_NAME -n $(oc project -q)
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):$SVC_ACCOUNT_NAME -n $(oc project -q)
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)

oc create secret generic $SECRET_NAME --from-literal=username=developer --from-literal=password=password --type=kubernetes.io/basic-auth -n $(oc project -q)
oc secrets link  $SVC_ACCOUNT_NAME $SECRET_NAME -n $(oc project -q)

#oc get --export template -n openshift -o json $TEMPLATE_NAME > $TEMPLATE_NAME.json
#oc process -f $TEMPLATE_NAME.json -p DB_USERNAME=$CUSTOM_DB_USERNAME -p DB_PASSWORD=$CUSTOM_DB_PASSWORD -p DB_DATABASE=$CUSTOM_DB_NAME | oc create -f - -n $(oc project -q)
oc process $TEMPLATE_NAME -p DB_USERNAME=$CUSTOM_DB_USERNAME -p DB_PASSWORD=$CUSTOM_DB_PASSWORD -p DB_DATABASE=$CUSTOM_DB_NAME | oc create -f - -n $(oc project -q)

5. app test
--------------------------------------------------------------------------------
oc logs -f bc/eap-app
oc logs -f dc/eap-app
oc get route -n eap72-mysql-s2i
oc describe route eap-app -n eap72-mysql-s2i
curl eap-app-eap72-mysql-s2i.apps.example.com

Reference URL)
https://github.com/jboss-container-images/jboss-eap-7-openshift-image/blob/eap72-dev/docs/templates/eap72-mysql-s2i.adoc
https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_application_platform/7.2/

JBoss Application / Web Server 설치 가이드

freedream — Fri, 26 Jul 2019 20:36:47 +0900

Red Hat Linux 환경에서 JBoss WebServer와 JBoss Application Server의 설치, 구성, 연동 방법에 대해서 정리해 보았습니다.

1. 설치 환경
----------------------------------------------------------------
● Red Hat Enterprise Linux7.5 (64bit)
    rhel-server-7.5-x86_64-dvd.iso
● Oracle JDK 1.8 (64bit)
   jdk-8u144-linux-x64.tar.gz
http://www.oracle.com/technetwork/java/javase/downloads/index.html
● Red Hat JBoss Web Server 3.0.3
   jws-application-servers-3.0.3-RHEL7-x86_64.zip
  jws-httpd-3.0.3-RHEL7-x86_64.zip
https://access.redhat.com/downloads/

2. JDK 설치
----------------------------------------------------------------
tar -xvzf jdk-7u80-linux-x64.tar.gz
mv jdk1.7.0_80 /usr/java/jdk1.7.0_80

rm -f /usr/java/latest
ln -s /usr/java/jdk1.7.0_80 /usr/java/jdk1.7

alternatives --install /usr/bin/java java /usr/java/jdk1.7/bin/java 10000
alternatives --install /usr/bin/javac javac /usr/java/jdk1.7/bin/javac 10000
alternatives --install /usr/bin/javaws javaws /usr/java/jdk1.7/jre/bin/javaws 10000
alternatives --install /usr/bin/jar jar /usr/java/jdk1.7/bin/jar 10000
alternatives --install /usr/bin/keytool keytool /usr/java/jdk1.7/bin/keytool 10000

alternatives --config java
alternatives --config javac

which java
readlink -f /usr/bin/javac /usr/java/jdk1.7.0_80/bin/javac

3. package 설치
----------------------------------------------------------------
yum -y update
yum clean all; yum repolist

yum remove tomcatjss
yum install apr apr-devel apr-util apr-util-devel apr-util-ldap elinks krb5-workstation mailcap

4. JBoss Web Server install
----------------------------------------------------------------
jws-application-servers-3.0.3-RHEL7-x86_64.zip
jws-httpd-3.0.3-RHEL7-x86_64.zip

5. JAVA_HOME, Tomcat, Apache User
----------------------------------------------------------------
● Creating a Tomcat User
groupadd -g 91 -r tomcat
useradd -c "Tomcat" -u 91 -g tomcat -s /bin/bash -r -p tomcat1234 tomcat

chown -R tomcat:tomcat tomcat
chmod -R u+X tomcat

vi /home/${user}/.bash_profile
JAVA_HOME=/usr/java/jdk1.7

PATH=$PATH:$JAVA_HOME/bin:$HOME/bin
export PATH

umask 022

# Enviroment Variable
export LANG=ko_KR.utf8
export EDITOR=vi
export TERM=vt100
set -o vi
PS1="$(whoami)@$(hostname)]"

# Alias
alias cdtomcat=’cd /app/jws-3.0/tomcat7’
alias cdhttpd=’cd /app/jws-3.0/httpd’

setclasspath.sh, catalina.sh
export JAVA_HOME=/usr/java/jdk1.7

● Creating an Apache User
groupadd -g 48 -r apache
useradd -c "Apache" -u 48 -g apache -s /bin/bash -r -p apache1234 apache

chown -R apache:apache httpd

※ Removing /Re-Adding SSL Support
JWS_HOME/httpd/conf.d/ssl.conf.disabled

● Enabling log4j Logging for Tomcat
cd JWS_HOME/extras/
cp log4j.properties $tomcat/lib/
cp log4j-eap6.jar $tomcat/lib/
cp tomcat-juli-adapters.jar $tomcat/lib/
cp tomcat-juli.jar $tomcat/bin/

6. HTTPD Configuring mod_jk
----------------------------------------------------------------
● JWS_HOME/httpd/conf.d/mod_jk.conf

● JWS_HOME/httpd/conf.d/workers.properties

worker.list=loadbalancer,jkstatus

worker.template.type=ajp13
worker.template.lbfactor=1
worker.template.ping_mode=A
worker.template.ping_timeout=5000
worker.template.prepost_timeout=5000
worker.template.connect_timeout=5000
worker.template.socket_timeout=300
worker.template.socket_keepalive=true
worker.template.connection_pool_size=64
worker.template.connection_pool_minsize=32
worker.template.connection_pool_timeout=60

# Set properties for node01
worker.tomcat01.reference=worker.template
worker.tomcat01.host=serverip1
worker.tomcat01.port=8009

# Set properties for node02
worker.tomcat02.reference=worker.template
worker.tomcat02.host=serverip2
worker.tomcat02.port=8109

worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=tomcat01,tomcat02
worker.loadbalancer.method=Session
worker.loadbalancer.sticky_session=True

worker.jkstatus.type=status

● JWS_HOME/httpd/conf.d/uriworkermap.properties
!/*.html=loadbalancer
!/*.css=loadbalancer
!/*.js=loadbalancer
!/*.gif=loadbalancer
!/*.png=loadbalancer
!/*.jpg=loadbalancer
!/*.swf=loadbalancer
/*=loadbalancer
!/server-status/*=loadbalancer

● JWS_HOME/httpd/conf/httpd.conf
Timeout 60
KeepAlive Off
MaxKeepAliveRequests 200
KeepAliveTimeout 3

DocumentRoot /app/htdocs
    DirectoryIndex index.html index.jsp
    ErrorLog "|/JWS_HOME/httpd/sbin/rotatelogs /JWS_HOME/httpd/logs/error_log.%Y%m%d 86400 +540"
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{Cookie}i\"" combined
    CustomLog "|/JWS_HOME/httpd/sbin/rotatelogs /JWS_HOME/httpd/logs/access_log.%Y%m%d 86400 +540" combined env=!dont_log
    JkMountFile /JWS_HOME/httpd/conf.d/uriworkermap.properties
</VirtualHost>

● JWS_HOME/httpd/conf.modules.d/00-mpm.conf

#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so => 주석 처리
LoadModule mpm_worker_module modules/mod_mpm_worker.so => 주석 해지

<IfModule worker.c>
StartServers         4
MaxRequestWorkers         1024
MinSpareThreads     128
MaxSpareThreads     256
ThreadsPerChild      64
MaxConnectionsPerChild  0
</IfModule>

7. Running the Apache HTTP Server Post-Installation Script
----------------------------------------------------------------
cd JWS_HOME/httpd
# ./.postinstall

chown root:root httpd
chmod u+s httpd

cp /JWS_HOME/httpd/sbin/apachectl /etc/init.d/httpd

vi /etc/init.d/httpd
#!/bin/sh
# chkconfig: 2345 90 90
# description: init file for Apache server daemon
# processname: /app/jws-3.0/httpd/sbin/apachectl
# config: /app/jws-3.0/httpd/conf/httpd.conf
# pidfile: /app/jws-3.0/httpd/logs/httpd.pid

chkconfig --add httpd
chkconfig --del httpd

chkconfig --list | grep httpd

systemctl enable httpd.service
systemctl start httpd.service

※ Compile SELinux Policies
# cd /etc/tomcat7/selinux/packages/tomcat7
# make -f /usr/share/selinux/devel/Makefile
# semodule -i tomcat7.pp

# cd /etc/tomcat8/selinux/packages/tomcat8
# make -f /usr/share/selinux/devel/Makefile
# semodule -i tomcat8.pp

● Add the firewall rules
firewall-cmd --permanent --add-service=http
firewall-cmd --reload
iptables -I INPUT -m state --state NEW -p tcp -m tcp --dport 80 -j ACCEPT
iptables-save > /etc/sysconfig/iptables

firewall-cmd --permanent --zone=public --add-port=80/tcp

# Mod Cluster firewall 설정
firewall-cmd --permanent --zone=public --add-port=23364/udp
firewall-cmd --permanent --zone=public --add-port=8888/tcp

/sbin/iptables -I INPUT 5 -p udp -d 224.0.1.0/24 -j ACCEPT -m comment --comment "mod_cluster traffic"
/sbin/iptables -I INPUT 6 -p udp -d 224.0.0.0/4 -j ACCEPT -m comment --comment "JBoss Cluster traffic"
/sbin/iptables -I INPUT 9 -p udp -s 192.168.1.0/24 -j ACCEPT -m comment --comment "cluster subnet for inter-node communication"
/sbin/iptables -I INPUT 10 -p tcp -s 192.168.1.0/24 -j ACCEPT -m comment --comment "cluster subnet for inter-node communication"
# /etc/init.d/iptables save
iptables-save > /etc/sysconfig/iptables

8. Tomcat 설정
----------------------------------------------------------------
● mod _cluster JBoss WebServer
# server.xml
--------------------------------------------------------------------
<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="true" stickySession="true" stickySessionForce="false" stickySessionRemove="true" />
<Engine name="Catalina" defaultHost="localhost" jvmRoute="worker01">

-Dorg.jboss.modcluster.container.catalina.status-frequency=6

## JBoss WebServer Worker Node with a Static Proxy List
# server.xml
--------------------------------------------------------------------
<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="false" stickySession="true" stickySessionForce="false" stickySessionRemove="true"/>

● Static ProxyList
--------------------------------------------------------------------
<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="false" stickySession="true" stickySessionForce="false" stickySessionRemove="true" proxyList="10.33.144.3:6666,10.33.144.1:6666"/>

● Heap Memory 설정
# catalina.sh
JAVA_OPTS="$JAVA_OPTS -server -Xms1024m -Xmx1024m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=512m -XX:MaxPermSize=512m"
JAVA_OPTS="$JAVA_OPTS -XX:+DisableExplicitGC -XX:-HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/logs/heapdump"
JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=2 -XX:-UseConcMarkSweepGC"
JAVA_OPTS="$JAVA_OPTS -XX:-PrintGC -XX:-PrintGCDetails -XX:-PrintGCTimeStamps -XX:-TraceClassUnloading -XX:-TraceClassLoading"
JAVA_OPTS="$JAVA_OPTS -Djava.awt.headless=true -Dfile.encoding=UTF-8"

-Xmx3550m -Xms3550m -Xmn2g -Xss128k -XX:+UseParallelGC -XX:ParallelGCThreads=20 -XX:+UseParallelOldGC

# server.xml
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener"/>

connectionTimeout="10000" redirectPort="8443" URIEncoding="UTF-8"
maxConnection="8192" maxThreads="250" maxHttpHeaderSize="8192" emptySessionPath="true"
enableLookups="false" acceptCount="100" disableUploadTimeout="true"/>

● datasource 설정

# /META-INF/context.xml

<Resource

name="jdbc/DsWebAppDB"
    auth="Container"
    type="javax.sql.DataSource"
    username="sa"
    password=""
    driverClassName="org.h2.Driver"
    url="jdbc:h2:mem:target/test/db/h2/hibernate"
    maxActive="8"
    maxIdle="4"/>

</Context>

# web.xml
<?xml version="1.0" encoding="UTF-8"?>

<web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">

<resource-env-ref>

<resource-env-ref-name>jdbc/DsWebAppDB</resource-env-ref-name>

<resource-env-ref-name>javax.sql.DataSource</resource-env-ref-name>

</resource-env-ref>

</web-app>

Jenkins Install on CentOS 7

freedream — Fri, 26 Jul 2019 20:27:10 +0900

1. Jenkins repository setting

-------------------------------------------------------------------------

sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo

sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key

# Individual Package Downloads

wget https://pkg.jenkins.io/redhat-stable/jenkins-2.89.4-1.1.noarch.rpm

https://pkg.jenkins.io/redhat-stable/

2. Java Install

-------------------------------------------------------------------------

yum list java*jdk-devel

yum install java-1.8.0-openjdk-devel.x86_64

ls -l /usr/bin/javac

/usr/bin/javac -> /etc/alternatives/javac

ls -l /etc/alternatives/javac

/etc/alternatives/javac -> /usr/lib/jvm/java-1.8.0/bin/javac

alternatives --config java

alternatives --config javac

java -version

# Set JAVA_HOME environment variable.

echo "export JAVA_HOME=/usr/lib/jvm/java-1.8.0" >> ~/.bashrc

echo "PATH=$PATH:$JAVA_HOME/bin:$HOME/bin" >> ~/.bashrc

echo "export PATH" >> ~/.bashrc

source ~/.bashrc

env | grep JAVA

JAVA_HOME=/usr/lib/jvm/java-1.8.0

3. Jenkins Install

-------------------------------------------------------------------------

1) yum update

#yum upgrade

yum clean all;yum repolist

2) install jenkins

yum install jenkins

off line)

wget https://pkg.jenkins.io/redhat-stable/jenkins-2.89.4-1.1.noarch.rpm

yum install jenkins-2.234-1.1.noarch.rpm

3) jenkins setting

# vim /etc/sysconfig/jenkins

JENKINS_PORT="8080"

JENKINS_ARGS="-Xmx2048m"

# vim /etc/rc.d/init.d/jenkins

candidates="

/usr/lib/jvm/java-1.8.0/bin/java

4) firewall setting

firewall-cmd --permanent --add-service=http

firewall-cmd --permanent --add-service=https

firewall-cmd --permanent --zone=public --add-port=8080/tcp

firewall-cmd --permanent --zone=public --add-port=80/tcp

firewall-cmd --permanent --zone=public --add-port=443/tcp

firewall-cmd --permanent --zone=public --add-port=22/tcp

firewall-cmd --reload

--------------------------------------

YOURPORT=8080

PERM="--permanent"

SERV="$PERM --service=jenkins"

firewall-cmd $PERM --new-service=jenkins

firewall-cmd $SERV --set-short="Jenkins ports"

firewall-cmd $SERV --set-description="Jenkins port exceptions"

firewall-cmd $SERV --add-port=$YOURPORT/tcp

firewall-cmd $PERM --add-service=jenkins

firewall-cmd --zone=public --add-service=http --permanent

firewall-cmd --reload

firewall-cmd --list-all

firewall-cmd --zone=public --list-all

4. start jenkins

-------------------------------------------------------------------------

1) start jenkins

systemctl enable jenkins

systemctl start jenkins

chkconfig jenkins on

systemctl status jenkins

2) jenkins console

http://$ip:8080/

cat /var/lib/jenkins/secrets/initialAdminPassword

3) jenkins 설정 파일

/var/lib/jenkins

/etc/default/jenkins

/etc/init.d/jenkins

※ administrator password

/var/lib/jenkins/home/secrets/initialAdminPassword

● To change attributes of the service

svccfg -s svc:/network/http:jenkins editprop

svcadm refresh svc:/network/http:jenkins

/lib/svc/manifest/network/jenkins-standalone.xml

jenkins log

tail -f /var/log/jenkins/jenkins.log

※ consult the log

/var/svc/log/network-http:jenkins.log

5. git, maven install

-------------------------------------------------------------------------

1) git install

yum install git

git --version

2) maven install

# yum install maven

#readlink -f /usr/bin/mvn /usr/share/maven/bin/mvn

wget http://apache.tt.co.kr/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz

tar -xvzf apache-maven-3.6.3-bin.tar.gz -C /opt

ln -s /opt/apache-maven-3.6.3 /opt/maven

# vim /etc/profile.d/maven.sh

export JAVA_HOME=/usr/lib/jvm/java-1.8.0

export M2_HOME=/opt/maven

export MAVEN_HOME=/opt/maven

export PATH=${M2_HOME}/bin:${PATH}

chmod +x /etc/profile.d/maven.sh

source /etc/profile.d/maven.sh

mvn --version (mvn -v)

#yum -y install wget git net-tools bind-utils iptables-services bridge-utils bash-completion kexec-tools sos psacct vim

6. gradle install

-------------------------------------------------------------------------

https://gradle.org/releases/

wget https://services.gradle.org/distributions/gradle-6.5-bin.zip

unzip -d /opt gradle-6.5-bin.zip

ln -s /opt/gradle-6.5 /opt/gradle

# Set gradle PATH environment

echo "PATH=$PATH:/opt/gradle/bin:$HOME/bin" >> ~/.bashrc

echo "export PATH" >> ~/.bashrc

source ~/.bashrc

gradle -v

https://pkg.jenkins.io/redhat-stable/

http://mirrors.jenkins-ci.org/redhat/

https://www.jenkins.io/doc/book/installing/

GitLab 설치하기

freedream — Fri, 26 Jul 2019 20:13:16 +0900

---------------------------------------------------------------------------
-- GitLab install
---------------------------------------------------------------------------

1. Install and configure the necessary dependencies
---------------------------------------------------------------------------
On CentOS 7 (and RedHat/Oracle/Scientific Linux 7), the commands below will also open HTTP and SSH access in the system firewall.

sudo yum install -y curl policycoreutils-python openssh-server
sudo systemctl enable sshd
sudo systemctl start sshd
sudo firewall-cmd --permanent --add-service=http
sudo systemctl reload firewalld

Next, install Postfix to send notification emails. If you want to use another solution to send emails please skip this step and configure an external SMTP server after GitLab has been installed.

sudo yum install postfix
sudo systemctl enable postfix
sudo systemctl start postfix

During Postfix installation a configuration screen may appear. Select 'Internet Site' and press enter. Use your server's external DNS for 'mail name' and press enter. If additional screens appear, continue to press enter to accept the defaults.

2. Add the GitLab package repository and install the package
---------------------------------------------------------------------------
Add the GitLab package repository.
# gitlab-ee
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.rpm.sh | sudo bash
# gitlab-ce
curl -s https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash

Next, install the GitLab package. Change `http://gitlab.example.com` to the URL at which you want to access your GitLab instance. Installation will automatically configure and start GitLab at that URL. HTTPS requires additional configuration after installation.
# gitlab-ee
sudo EXTERNAL_URL="http://gitlab.example.com" yum install -y gitlab-ee

# gitlab-ce
sudo EXTERNAL_URL="http://gitlab.example.com" yum install -y gitlab-ce

3. Browse to the hostname and login
---------------------------------------------------------------------------
On your first visit, you'll be redirected to a password reset screen. Provide the password for the initial administrator account and you will be redirected back to the login screen. Use the default account's username root to login.

See our documentation for detailed instructions on installing and configuration.

4. Set up your communication preferences
---------------------------------------------------------------------------
Visit our email subscription preference center to let us know when to communicate with you. We have an explicit email opt-in policy so you have complete control over what and how often we send you emails.

Twice a month, we send out the GitLab news you need to know, including new features, integrations, docs, and behind the scenes stories from our dev teams. For critical security updates related to bugs and system performance, sign up for our dedicated security newsletter.

IMPORTANT NOTE: If you do not opt-in to the security newsletter, you will not receive security alerts.

5. gitlab restart
---------------------------------------------------------------------------
systemctl restart gitlab-runsvdir
systemctl status gitlab-runsvdir

systemctl list-units --type=service
systemctl list-units --type=service --state=running
systemctl list-unit-files |grep docker

https://about.gitlab.com/installation/#centos-7

---------------------------------------------------------------------------
-- # gitlab offline install
---------------------------------------------------------------------------
1. gitlab rpm download
---------------------------------------------------------------------------
https://packages.gitlab.com/gitlab/gitlab-ce
# latest version
https://packages.gitlab.com/gitlab/gitlab-ce/packages/el/7/gitlab-ce-11.3.0-ce.0.el7.x86_64.rpm
https://packages.gitlab.com/gitlab/gitlab-ee/packages/el/7/gitlab-ee-11.3.0-ee.0.el7.x86_64.rpm

2. 내려받은 rpm을 서버에 upload하고 설치합니다.
---------------------------------------------------------------------------
rpm -ivh gitlab-ce-11.3.0-ce.0.el7.x86_64.rpm

3. gitlab 설치
---------------------------------------------------------------------------
local dns에 해당 domain을 등록합니다.
(또는 /etc/host파일에 domain등록)

sudo EXTERNAL_URL="http://gitlab.example.com" yum install -y gitlab-ee

4. console login & 사용자 등록
---------------------------------------------------------------------------
http://gitlab.example.com

5. gitlab 재기동
---------------------------------------------------------------------------
systemctl restart gitlab-runsvdir
systemctl status gitlab-runsvdir

6. gitlab external web port change
---------------------------------------------------------------------------
sudo -e /etc/gitlab/gitlab.rb
# nginx: 90, unicorn: 8090로 변경하는 경우

external_url 'http://gitlab.example.com:90'

unicorn['worker_timeout'] = 60
unicorn['listen'] = '127.0.0.1'
unicorn['port'] = 8090   # 8080 => 8090

sudo gitlab-ctl reconfigure
cat /var/opt/gitlab/gitlab-rails/etc/gitlab.yml
production: &base
  #
  # 1. GitLab app settings
  # ==========================

  ## GitLab settings
  gitlab:
    ## Web server settings (note: host is the FQDN, do not include http://)
    host: gitlab.example.com
    port: 90
    https: false

gitlab-ctl stop
gitlab-ctl start

netstat -anp | grep :90
netstat -anp | grep :8090

# console 로그인
http://gitlab.example.com:90

https://docs.gitlab.com/omnibus/settings/configuration.html
https://docs.gitlab.com/omnibus/settings/nginx.html

7. Manually configuring HTTPS
---------------------------------------------------------------------------
/etc/gitlab/gitlab.rb
external_url "https://gitlab.example.com:3443"

sudo mkdir -p /etc/gitlab/ssl
sudo chmod 700 /etc/gitlab/ssl
sudo cp gitlab.example.com.key gitlab.example.com.crt /etc/gitlab/ssl/

# SSL firewall-cmd
sudo firewall-cmd --permanent --add-service=https
sudo systemctl reload firewalld

# Redirect HTTP requests to HTTPS
external_url "https://gitlab.example.com:3443"
nginx['redirect_http_to_https'] = true

# For GitLab
nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.example.com.crt"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.example.com.key"

# Setting the NGINX listen port
nginx['listen_port'] = 8081

sudo gitlab-ctl reconfigure

systemctl restart gitlab-runsvdir
systemctl status gitlab-runsvdir sudo gitlab-ctl restart
sudo gitlab-ctl stop

systemctl restart gitlab-runsvdir
systemctl status gitlab-runsvdir

systemctl list-units --type=service
systemctl list-units --type=service --state=running
systemctl list-unit-files |grep docker

8. Supporting proxied SSL
---------------------------------------------------------------------------
vi /etc/gitlab/gitlab.rb

registry_external_url 'https://registry.example.com'
registry_nginx['listen_port'] = 80
registry_nginx['listen_https'] = false

# Setting HTTP Strict Transport Security
nginx['hsts_max_age'] = 31536000   # default: 1 year
nginx['hsts_include_subdomains'] = false

---------------------------------------------------------------------------
-- Gitlab Install (OpenShift)
---------------------------------------------------------------------------

1. private docker registry Image Import
---------------------------------------------------------------------------
docker pull gitlab/gitlab-ce
docker tag gitlab/gitlab-ce:latest registry.example.com:5000/gitlab/gitlab-ce:latest
docker push registry.example.com:5000/gitlab/gitlab-ce:latest

2. push images from the integrated container registry directly
---------------------------------------------------------------------------
docker login -p $(oc whoami -t) -e unused -u unused docker-registry-default.apps.example.com
docker tag registry.example.com:5000/gitlab/gitlab-ce:latest docker-registry-default.apps.example.com/openshift/gitlab-ce:latest
docker push docker-registry-default.apps.example.com/openshift/gitlab-ce:latest

3. Updating and Tagging Existing Images
---------------------------------------------------------------------------
oc import-image registry.example.com:5000/gitlab/gitlab-ce:latest --insecure=true --confirm -n openshift
oc tag gitlab-ce:latest gitlab-ce:12.1 -n openshift

4. Create project
---------------------------------------------------------------------------
oc new-project gitlab
oc adm policy add-scc-to-user anyuid -z default -n gitlab
oc adm policy add-scc-to-user anyuid system:serviceaccount:gitlab:gitlab-ce-user

5. Create serviceaccount and secret
---------------------------------------------------------------------------
oc create serviceaccount gitlab-sa -n $(oc project -q)
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):gitlab-sa -n $(oc project -q)
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)

oc create secret generic gitlab-secret --from-literal=username=user --from-literal=password=1234 --type=kubernetes.io/basic-auth -n $(oc project -q)
oc secrets link  gitlab-sa gitlab-secret -n $(oc project -q)

6. Create a new application
---------------------------------------------------------------------------
● CLI
oc new-app registry.example.com:5000/gitlab/test-gitlab:1.0 --name=gitlab-ce -n gitlab
oc expose svc/gitlab-ce --hostname=gitlab.example.com --port 80 -n gitlab
oc get route gitlab-ce -n gitlab

or
oc new-app registry.example.com:5000/gitlab/gitlab-ce:1.0 -o yaml -n gitlab > gitlab-ce.yaml
vi gitlab-ce.yaml
oc create -f gitlab-ce.yaml -n gitlab

● console
- console > gitlab project > Add to Project > Deploy Image
- Namespace: openshift
- Image Name: registry.example.com:5000/gitlab/test-gitlab:1.0
deploy

8. PV(Persistent Volume), PVC(Persistent Volume Claim)
---------------------------------------------------------------------------
volume name          Container location     Usage
---------------------------------------------------------------------------
gitlab-ce-volume-1    /etc/gitlab           For storing application data
gitlab-ce-volume-2    /var/log/gitlab       For storing logs
gitlab-ce-volume-3    /var/opt/gitlab       For storing the GitLab configuration files

환영합니다!

freedream — Tue, 21 May 2019 23:15:13 +0900

PaaS(platform as a services), DevOps, digital transformation, Middleware, opensource에 대한 글을 게재하려 합니다.