Using Perl to read from elasticsearch

A perl script to read into elasticsearch

use Search::Elasticsearch; use URI::Escape; use DateTime;

$dt = DateTime->now; $start_timestamp = join ' ', $dt->ymd, '00:00:00'; $end_timestamp = join ' ', $dt->ymd, '23:59:59';

my $client = "something";

my $es = Search::Elasticsearch->new(trace_to => ['File','/var/log/perl/log-'.$start_timestamp.'.log'],nodes=>['http://10.9.8.x:9200/']);

my $scroll = $es->search(index => 'logstash-*',body => {"_source" => ["Name","syslogHostName"],"query" => { "match" => { "ClientName.raw" => "$client" } } }, size => 3000);

my @results = @{ $scroll->{hits}{hits} }; print "Total number of hosts: ".scalar @results."\n\n"; for (my $i=0 ; $i < (scalar @results); $i++ ) { print $results[$i]->{_source}->{syslogHostName}."\n"; }

Elasticsearch cluster setup - Three node cluster

The following changes are required in elasticsearch.yml

Node1

cluster.name: lab

node.name: node-name

bootstrap.memory_lock: true

bootstrap.system_call_filter: false

network.host: hostname

http.port: 9200

node.master: true

node.data: true

discovery.zen.ping.unicast.hosts: ["x.x.x.x", "x.x.x.x"]

discovery.zen.minimum_master_nodes: 2

Node2

cluster.name: lab

node.name: node-name

bootstrap.memory_lock: true

bootstrap.system_call_filter: false

network.host: hostname

http.port: 9200

node.master: true

node.data: true

discovery.zen.ping.unicast.hosts: ["x.x.x.x", "x.x.x.x"]

discovery.zen.minimum_master_nodes: 2

Node3

cluster.name: lab

node.name: node-name

bootstrap.memory_lock: true

bootstrap.system_call_filter: false

network.host: hostname

http.port: 9200

node.master: true

node.data: true

discovery.zen.ping.unicast.hosts: ["x.x.x.x", "x.x.x.x"]

discovery.zen.minimum_master_nodes: 2

Further advice to setup coordination node in the cluster

It is not feasible to have a coordination node(client node) in a 4 nodes cluster. To have such provision we need to have 5 nodes cluster; I will explain the same with the following details.

The master/data node architecture is configured with three important parameters they are:

1. node.master
2. node.data
3. discovery.zen.minimum_master_nodes

The first two parameters say whether a node is master eligible or not by setting node.master to true. The third parameter is an important factor to elect a new master in case the acting master goes down.

If a cluster has three eligible master nodes then the value of minimum_master_nodes is calculated as (3/2)+1 = 2. So in a 4 node cluster there should be four master eligible nodes and the minimum_master_nodes value should be equal to 3. In that case we cannot have a coordination node which should not be a master eligible node. It is always a recommended practice to have the number of nodes in odd series than an even series. So if we consider 5 nodes cluster then we will have four master eligible nodes and one coordination node (5/2)+1 = 3.

Significance of discovery.zen.minimum_master_nodes:

If a master goes down in a cluster this value governs the election of new master. Unless the value is met, for example if the value is three unless there are three master eligible nodes a new master will not be elected. This is to avoid a split brain issue. A split brain problem may occur if any of the data nodes goes out of cluster due to a network outage for example, then it will not promote itself to become master because there is only one master eligible node. If this is not controlled then the node which is not connected will promote itself as a master and it will cause data loss when put back to the cluster. To avoid such issues this value is considered significant.

So a 5 nodes cluster can be organized as

1. Master/Data node – a primary master and master eligible node
2. Master/Data node – a master eligible node
3. Master/Data node – a master eligible node
4. Master/Data node – a master eligible node
5. Coordination(client) node – not a master eligible node

The kibana and logstash can be connected to this coordination node which will act as a load balancer for the elasticsearch cluster.

Chef administration

Backup and restore
chef-server-ctl backup --yes
it will bring down chef server and then take backup

chef-server-ctl restore /path/to/backup

Docker notes

To start a service when a container starts,

use entrypoint

ENTRYPOINT service elasticsearch start && bash

A sample knife.rb file with exception for ssl mode

current_dir = File.dirname(__FILE__)

log_level                :info

log_location             STDOUT

node_name                'rajagopalan'

client_key               '/root/chef-repo/.chef/rajagopalan.pem'

validation_client_name   'hexaware'

validation_key           '/root/chef-repo/.chef/hexaware-validator.pem'

chef_server_url          'https://api.chef.io/organizations/ORG_NAME'

cache_type               'BasicFile'

cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )

cookbook_path            ['#{current_dir}/../cookbooks']

Vi  ~/.gemrc

Add this line to bypass ssl check

:ssl_verify_mode: 0

Add this line to knife.rb to exclude ssl check while executing knife ec2 server create

Excon.defaults[:ssl_verify_peer] = false

Fix - ERROR: Server returned error 500 for https://127.0.0.1/users/ - Chef

If the following error is faced in chef-server, version 12, then do the following to fix the issue.


ERROR: Server returned error 500 for https://127.0.0.1/users

open the file /opt/opscode/embedded/cookbooks/private-chef/templates/default/oc_erchef.config.erb in vi editor and go to line 220.

Replace the following line :

{s3_url, "<%= node['private_chef']['nginx']['x_forwarded_proto'] %>://<%= @helper.vip_for_uri('bookshelf') %>"},

with

{s3_url, "https://private-chef.opscode.piab:4000"},

and then run chef-server-ctl reconfigure.

Reason and Solution:

nginx will listen on port 4000 for HTTPS connections and not the default port of 443.

During cookbook uploads, the opscode-erchef service talks to bookshelf via the s3_url in its configuration file (/var/opt/opscode/opscode-erchef/etc/app.config). This configuration file is rendered via a template(opscode-omnibus/files/private-chef-cookbooks/private-chef/templates/default/oc_erchef.config.erb), a portion of which looks like:

{s3_url, "<%= node['private_chef']['nginx']['x_forwarded_proto'] %>://<%= @helper.vip_for_uri('bookshelf') %>"},
Thus, the rendered configuration file will have an s3_url like:

{s3_url, "https://private-chef.opscode.piab"},
Given this configuration, erchef will attempt to contact erchef on port 443, the default HTTPS port. Unfortunately, nothing is listening on 443, the request to bookshelf fails and erchef returns a 500 to the user.

An astute user may attempt to set bookshelf['vip'] in private-chef.rb to something like:

bookshelf['vip'] = 'private-chef.opscode.piab:4000'

Reference : https://github.com/chef/chef-server/issues/50

Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused

run mongod process with the dbpath parameter

mongod --dbpath /home/mongo/data/db

create the path if it does not exits.