RSS

My recent experience with AZ-104 exams (2024)

Well I have to get my AZ-104 exams early this year, and as part of industry practice, went to exam sites, Youtube and downloaded pdfs to study for the exam. Yes, you don’t even need a login to azure.com to pass the exams. I took the exams early this year in 2024, which means that there are some newer topics that most exam sites and Youtuber will not cover.

In summary, only about 30% of what I studied appeared in the exam, the rest are new to me. For example, there is a question about setting up a VM as a router to force traffic through it and another about DNS name label reuse, which I never heard of until that day! This is an open book exam, meaning you can access learn.microsoft.com website to look for answers. This is NOT a blessing because it’s not like a google search. For example if you search for “storage account sku”, you expect that the top result would give you that page exactly. Instead, the results seem pretty random to me and you are required to make a good guess as to which page may hold the answer.

My advice for those going for the exams is not to focus on answering all questions correctly in one go. If you get stuck on a question, mark it review, answer to the best of your ability and move on to the next. Even though you have 2 hours, the clock seems to tick much faster during exams. What you want to achieve is to complete answering everything with 45 to 30 mins of time left. Now you go through all the questions for review and do a search on the MS learn website to figure out your answer. I think most importantly, you need to have a good grasp of azure public cloud and a good understanding about virtual computing and networks. This will make your exams an easier affair.

Anyway, I passed on the first try.

 
Leave a comment

Posted by on January 23, 2024 in Cloud

 

Tags: , , , , ,

Using Ansible playbooks for security or baseline compliance

Ansible playbooks is commonly used for configuration management, but we are also using it to ensure security or baseline compliance. This could be operating systems, appliances or even switches. The main difference for the playbook is that it only checks configuration and reports status, it doesn’t make changes. Some may ask why can’t you apply role to those inventories instead, in that way your system will always be compliant? Well, you want the playbook to run on schedule and report the findings. In a large controlled environment, you do not want changes to be made to your systems unless they are planned. Obviously, if the risk and impact is properly managed and negotiated with your change and risk teams, it is still possible.

The main feature of such a playbook, different from most, is that it’s used for reporting and outputting the findings into either another system or a file in which the report can be read. You cannot depend on the ansible.log for such information.

For this, I have created 3 tasks files for such playbooks:

  • To prune old log files
  • To output results to a log file
  • To output result to syslog (in my case vRealize Log Insight)

prune_logfiles.yml

This file is required so that you don’t fill up disk with log files every run. It just a basic tasks to remove 10 days old files

# pre-requsite vars
# - logpath
# - logpattern

- name: Get old log files
  find:
    paths: "{{ logpath }}"
    age: 10d
    patterns: "{{ logpattern }}"
  register: oldfiles
  run_once: true
  tags: always

- name: Prune old logs
  file:
    path: "{{ item.path }}"
    state: absent
  with_items: "{{ oldfiles.files }}"
  loop_control:
    label: "{{ item.path }}"
  run_once: true
  tags: always

write_logfile.yml

This task will write a structured message to a log file. You can set up a CSV or tab delimited log, but choose to use “|” as delimited.

# to use this task, you need to define the follow vars before you call 
# - logfile: full path of output file
# - logsubject: title of the event
# - logstatus: PASS|FAIL|SUCCESS, etc
# - logdesc: description of subject
# - logmsg: the message for this event

- name: Write to logfile 
  vars:
    logtext: "{{ lookup('pipe', 'date --iso-8601=seconds') + '|' + inventory_hostname + '|' + logdesc + '|' + logstatus + '|' + logmsg }}"
  lineinfile:
    path: "{{ logfile }}"
    line: "{{ logtext }}"
    insertafter: EOF
    create: true
  delegate_to: localhost
  tags: always

write_vrli_log.yml

Lastly, this task will write a structure syslog message to your syslog server. In this case, we are writing to a vRealize Log Insight syslog server using its api. Obviously, you need to make sure all variables especially the syslog are defined before executing. The HOSTNAME task is to ensure that I report in the syslog message itself which host the log is coming from.

# to use this task, you need to define the follow vars before you call 
# - logserver: syslog server
# - logsubject: title of the event
# - logstatus: PASS|FAIL|SUCCESS, etc
# - logdesc: description of subject
# - logmsg: the message for this event

- name: Get ansible hostname
  ansible.builtin.shell: echo $HOSTNAME
  register: result
  delegate_to: localhost
  tags: always
 
- name: Send event to syslog
  vars:
    logurl: "https://{{logserver}}:9543/api/v1/events/ingest/ansible-playbook"
    thishost: "{{ result.stdout }}"
  ansible.builtin.uri:
    url: "{{ logurl }}"
    method: POST
    body:
      events:
        - fields:
            - name: "appname"
              content: "ansible"
            - name: "status"
              content: "{{ logstatus }}"
          text: >
            "{{thishost}} ansible
            subject=[{{logsubject}}] host={{inventory_hostname}}
            status={{logstatus}} desc=[{{logdesc}}] msg=[{{logmsg}}]"
    body_format: json
    headers:
      Content-Type: application/json
    validate_certs: false
  delegate_to: localhost
  tags: always
 

main.yml

Here is a sample of an NSX-T compliance baseline playbook. The main playbook itself has two plays. The first is just used to clear old logs. We only need to run it once and run it against the localhost, where the logs are kept.

The second play is the main playbook. This is not a full playbook; I sniped a short sample to show you how it’s done but in general this is how the structure looks like. I sort of adopted parts of the structure based on what I found in the STIG ansible playbooks.

You will notice that I used “serial:1” in the run. Since this is for compliance reporting, you want the logs and results to be serially created, so that they can be read easily raw. Of course, if you are dealing with a large number of inventories, you may want to remove it; to read the report, just import them into Excel (for example) and sort the data accordingly.

You can see one task is used to list all the compliance items. Keeping them all in this section makes it easier to see in one glance what items are being checked. Any values or lists that may be used to check pass/fail conditions are defined here, prefixed with the item reference.

Each compliance item consists of at least 3 tasks (it can be more if required) starting with a task to get the values you need and register it into a variable. The next two tasks simply write pass or fail statements to log files and syslog based on positive or negative conditions using “when”. The way the cluster of tasks are written is to make them easy to copy and paste with minimal changes. Tags is used generally during testing of your playbook.

- name: Clear out old log files
  hosts: localhost
  
  vars:
    logpattern: 'nsxt_security_baseline_check'

  tasks:
  - name: Task to prune logfiles
    include_tasks: ./common/prune_logfiles.yml
    
- name: NSX-T security baseline check
  hosts: nsxt
  serial: 1

  vars:
     logpath: /mypath/log
     date: "{{ lookup('pipe', 'date +%Y%m%d') }}"

  tasks:

  - name: Initialize vars
    set_fact:
      logfile: "{{ logpath }}nsxt_security_baseline_check_{{ inventory_hostname }}_{{ date }}.log"
    delegate_to: localhost
    tags: always

  - name: define logging facts
    set_fact:
      logserver: "mysyslogserver1.com"
      logsubject: "NSX-T security baseline compliance"
    delegate_to: localhost
    run_once: true
    tags: always

  - name: define security baseline vars
    set_fact:
      nsxt001: "NSXT-001: Ensure Custom certificate is used"
      nsxt002: "NSXT-002: Ensure syslog server is correct"
	  nsxt002_syslog_servers: ["syslog1"]
      nsxt003: "NSXT-003: Ensure NTP servers are correct"
      nsxt003_ntp_servers: ["ntp1","ntp2"]       
      nsxt004: "NSXT-004: Ensure backup is configured"
      nsxt005: "NSXT-005: Ensure SNMP v2 is disabled"
      nsxt006: "NSXT-006: Password length must be at least 15 chars"
      nsxt006_minlen: 15
     delegate_to: localhost
    run_once: true
    tags: always

...

  - name: "{{ nsxt005 }}"
    vars:
      nsxt_api: "/api/v1/node/services/snmp?show_sensitive_data=false"
    ansible.builtin.uri:
      url: "https://{{inventory_hostname}}{{nsxt_api}}"
      user: 
      password: 
      method: GET
      force_basic_auth: true
      validate_certs: false
      headers:
        Content-Type: application/json
    register: snmp_config
    delegate_to: localhost
    tags: NSXT-005

  - name: Write to logs (PASS)
    vars:
      logmsg: "Configuration is correct"
      logstatus: "PASS"
      logdesc: "{{ nsxt005 }}"
    include_tasks: "{{ item }}"
    loop:
      - ./common/write_logfile.yml
      - ./common/write_vrli_log.yml    
    when: not snmp_config.json.service_properties.v2_configured
    tags: NSXT-005

  - name: Write to logs (FAIL)
    vars:
      logmsg: "Configuration is NOT correct"
      logstatus: "FAIL"
      logdesc: "{{ nsxt005 }}"
    include_tasks: "{{ item }}"
    loop:
      - ./common/write_logfile.yml
      - ./common/write_vrli_log.yml          
    when: snmp_config.json.service_properties.v2_configured
    tags: NSXT-005
	
....

Other stuff

You may notice that the playbook tasks are littered with “tags: always”. This is important if you are using tags and you want to ensure that tasks are run when tags are used. This is especially important in included task files. Without them, the tasks in the task file will not run even if the main task is tagged to run.

 
Leave a comment

Posted by on July 27, 2023 in ansible

 

Tags: , , , ,

Ansible: Updating a value in a list in a dictionary

Its been awhile since I posted, mostly because work has been very busy. Anyway, I have been working a lot more with ansible recently and it is both easy to start and very steep learning curve with regardless to more complex issue. This one sounds straight forward with any reasonable code py, ps1, etc, but it too the chunk of 3-4 days of endless searching and test to get it working finally.

One of the tasks I am looking to do is to query NSX-T manager and disable TLSv1.1 if it’s been enabled.

  - name: Call NSXT
    vars:
      api: "/api/v1/cluster/api-service"
    ansible.builtin.uri:
      url: "https://server01{{ api }}"
      user: user1
      password: pwd1
      method: GET
      force_basic_auth: true
      validate_certs: false
      headers:
        Content-Type: application/json
    delegate_to: localhost
    register: result

The returning result.json looks like this (I clipped of other value to keep this short)

{
"global_api_concurrency_limit": 199,
"client_api_rate_limit": 100,
"client_api_concurrency_limit": 40,
"connection_timeout": 30,
"redirect_host": "",
"protocol_versions": [
  {"enabled": true, "name": "TLSv1.1"},
  {"enabled": true, "name": "TLSv1.2"}
]
}

True, I can create a standard template for the policy from this and push it to NSX-T manager regardless, but the danger of that is that if there are new values, e.g. TLSv1.3, that template will not contain the new values and it will be removed upon application. Nobody will know about this util one day something breaks on the web front.

The solution that I found (but also thanks to U880D on StackExchange for pointing me the way) is the below. This will update “enabled” field to false only for name=TLSv1.1

 - ansible.utils.update_fact:
      updates:
        - path: "result.json.protocol_versions[{{item}}].enabled"
          value: false
    loop: "{{ range(result.json.protocol_versions|length) }}"
    when: result.json.protocol_versions[{{item}}].name == "TLSv1.1"
    register: updated

Why did the solution take so long to derive? Mainly because of my newbie status and the confusion between a list and dictionary. The problem for me was that I was not sure of how I can update a value within a list within a dictionary. I searched for “nested dictionary”, blah, blah, but never found the answer. It was only after I realized the protocol_versions contains a list that my search became fruitful.

How does the above code work?

  • First we setup a loop for the list upper bounds of protocol_versions
  • “when” condition ensures that update only happens when the match occurs
  • With the correct index to the list, we can then update the correct value, register that and use it to update NSX-T.
 
Leave a comment

Posted by on July 21, 2023 in ansible

 

Tags: , ,

PowerNSX: Compare config of two edges

Recently, I had to perform a scale out operation for NSX edges managed by my team. We are running with 4 Edges and needed to scale out another 4 more to improve overall traffic performance.

One of the challenge for me is “How do I know if the configuration is correct for each new Edge that I create?”. This is why I wrote a quick script to allow me to compare the configuration a sample Edge already in used with the new edge that is created.

The script Compare-NSXEdge.ps1 simple compares settings with the Edges that I feel that must be the same. The script doesn’t check for correctness of your configuration. If you configured the sample Edge wrongly and the Edge that you are comparing with is also wrong, the script will say is green. This is very raw cut of the script; I just needed it to do the job during my change. You can add other settings as per your own requirements.

You can download the script from here -> https://github.com/kwongheng/PowerNSXScripts/blob/main/Compare-NSXEdge.ps1

 
Leave a comment

Posted by on September 5, 2022 in Operations, PowerNSX, Scripts

 

Tags: , , , ,

Nuances running powershell with Ansible

Recently, we got to use Ansible as part of automation effort within our team. One of my item was to work on NSX compliance. Basically, we have a set of configuration in our environment for NSX edges and logical routers that we would like to enforce, e.g. ICMP redirect, syslogs, SSH being disabled, etc. It quite a huge learning curve for someone mostly writing on Windows and now need to use Ansible with Linux and Python. Here are some of my learnings:

Considerations for parallel script runs

  • By default, ansible will attempt to run the same task for all your hosts at once. This means that the same script is run in parallel.
  • Each script needs to load PowerNSX module, as a result some runs fail because of loading error due to file locks when PowerNSX module tries to initialize. One way around this is use “serial” to control your batches, but this can take a long time if you have a lot of objects and scripts. Another way is describe below but in your main file.
  • The scripts which I use also output to a log file. The logic first checks if the file exists if not it will create it. Again, racing condition can cause the same file to be create at the same time. To overcome that, I added a random wait timer in milliseconds between 0-1000 in the function to resolve this issue.

Returning status from pwsh script to ansible

  • If you are using ansible plugin, then status of each task run is already handled by the plugin. However, this is not the case with running pwsh. Even if your script fails, it will still return “OK” status.
  • In order to communicate to ansible error or changed status, you need to use “changed_when” and “error_when” variables. You use write-host and write-error to communicate status of the execution through “register” variable.
    - name: ICMP redirect is disabled on Edge      
      delegate_to: localhost
      command: pwsh ../../files/nsx/MyScript -vcname {{ vcenter }} -vc_user {{ vcenter_username }} -vc_pwd {{ vcenter_password }} -objectname {{ inventory_hostname }} -logfile {{ logfile }}
      register: pwsh_output
      failed_when: 
        - pwsh_output.stderr != ''
      changed_when:
        - '"[UPDATE]" in pwsh_output.stdout'

Run_once + Serial in same playbook

  • Not related to pwsh, rather a lesson from an ansible newbie.
  • My original assumption was that run_once will be honor. However, once you add serial to your playbook, task with run_once runs for every batch, not just once
  • The solution is to have two plays in your playbook, once for run_once task and another for serial/batch tasks.
 
Leave a comment

Posted by on June 18, 2022 in powershell

 

PowerNSX: How to add/remove NSX Edge BGP filters

PowerNSX is a very wonderful tool for automating NSX, but it lacks the ability to add/remove elements from the objects. One example is BGP filters. I have seen many people asked about it, but not presented with proper solutions. This is also something that I had been struggling to understand especially being an XML code newbie. I managed to find some sample codes in someone’s github code base unrelated to bgp filters, so credit to whoever helped point me in the right direction. I then took it forward and did further research on how to get it done.

The codes can be used for any other elements in NSX Edge that you want to add/remove as long as you understand the basic principles and XML structure. Another good example is enabling syslog on NSX Edges, you cannot just set syslog to enable without adding the syslog related elements and hence these codes are required.

Add BGP filters

#first you want to grab the bgp object, assuming edge-3 is the object of interest
$edgebgp = Get-NsxEdge -objectId edge-3 | Get-NsxEdgeRouting | Get-NsxEdgeBgp

#Now you what to examine the structure of Bgp
#You know that each bgpfilter has direction, action and network elements
$edgebgp | format-xml

<bgp>
  <enabled>true</enabled>
  <localAS>63000</localAS>
  <localASNumber>63000</localASNumber>
  <bgpNeighbours>
    <bgpNeighbour>
      <ipAddress>10.10.10.12</ipAddress>
      <remoteAS>63001</remoteAS>
      <remoteASNumber>63001</remoteASNumber>
      <weight>60</weight>
      <holdDownTimer>3</holdDownTimer>
      <keepAliveTimer>1</keepAliveTimer>
      <password>********</password>
      <bgpFilters>
        <bgpFilter>
          <direction>out</direction>
          <action>deny</action>
          <network>10.10.12.0/24</network>
        </bgpFilter>
        <bgpFilter>
          <direction>out</direction>
          <action>deny</action>
          <network>10.10.13.0/24</network>
        </bgpFilter>
...

#Now you start building the bgpfilter element 
#Assuming you want to add a new network deny out
$BgpFilter = $Edgebgp.OwnerDocument.CreateElement("bgpFilter")
Add-XmlElement -xmlRoot $BgpFilter -xmlElementName "direction" -xmlElementText "Out"
Add-XmlElement -xmlRoot $BgpFilter -xmlElementName "action" -xmlElementText "deny"
Add-XmlElement -xmlRoot $BgpFilter -xmlElementName "network" -xmlElementText "10.10.30.0/24"

#Now you grab the neighbour that you want to add this.
#You can repeat the same for another neighbour
$bgpneighbour = $edgebgp.bgpNeighbours.SelectNodes('//bgpNeighbour[ipAddress="10.10.10.12"]')
$bgpneighbour.bgpFilters.AppendChild($BgpFilter )

#You can now verify if its added in the entire object, assuming position is 0
$edgebgp.bgpNeighbours.bgpNeighbour[0].bgpFilters.bgpFilter

Remove BGP Filter

#On the same note, if we want to remove the rule for 10.10.13.0/24
$bgpneighbour = $edgebgp.bgpNeighbours.SelectNodes('//bgpNeighbour[ipAddress="10.10.10.12"]')
$node = $bgpneighbour.bgpFilters.SelectSingleNode('//bgpFilter[network = "10.10.13.0/24"]')

#Now you can remove this child node
$bgpneighbour.bgpFilters.RemoveChild($node)

#You can now verify if its removed, assuming position is 0
$edgebgp.bgpNeighbours.bgpNeighbour[0].bgpFilters.bgpFilter
 
Leave a comment

Posted by on June 18, 2022 in PowerNSX, powershell

 

PS: Breaking long string to fixed width

Was working on a KB from VMware and had to pick up a certificate signature string and covert it to PEM format in 64 character width. I am sure that if search hard enough in the net, there are better examples. Basically, it takes in the long strong, grabs 64 characters and puts into a string array and remove them from the original string. This repeats until the string is 64 characters or less

$longstr = gc .\certstring.txt ; 
$PEMstr = @() ; 
do { 
  $PEMstr += $longstr.Substring(0,64) ; 
  $longstr = $longstr.Remove(0,64) } 
while ( $longstr.length -gt 64 ) ; 
$PEMstr += $longstr

#The above code will convert a single long string to 64 character width
$PEMstr

MIIDyjCCArKgAwIBAgIJAPjk7qtCv+CIMA0GCSqGSIb3DQEBCwUAMIGgMQswCQYD
…………
P2YtUKWLPhiWhiA183LZFqeZRJ0ih6JaLkvmfwy/RwrDzyZ6jhtMDGMhBTzpQQID

 
Leave a comment

Posted by on April 2, 2021 in powershell

 

PS: Script to reboot ESXi hosts after changes

I wrote this script as there a countless times where we need to make changes to hosts and reboot after those changes. Not all of those changes can be done with update manager and hence without a script manual reboot is required, for example, adjusting some kernel or memory limit values.

As this is a real world script, i.e. you don’t want a runaway loop which reboots all your hosts at once, there are many logic checks before it moves on to the next step. If all fails, it will exit the script so that you can fix the issue host(s).

The script will set the host first to maintenance mode, make the changes and reboot the host. Only after that host online will it move on to the next host. Currently, I have only coded for 1 host at a time, so in a large cluster, it make take some time to completed. I hope to have some time to improve on the script to allow nth hosts to be worked on at one time.

You may get it from github here ->

kwongheng/Template-RebootESXiHosts-AfterUpdate (github.com)

 
Leave a comment

Posted by on March 27, 2021 in powercli

 

Tags: , ,

PowerCLI: Backup VDswitch and VDportgroups

Here is a quick set of scripts to export and backup your VDswitches. It is pretty straight forwards. It dumps VDswitch exports with and without port groups and the VDportgroups into their own folders. Then zip up each folder and finally create an archive from that 3 folders into a single backup zip. You can delete the folders and the other zip files after it ran successfully

#first connect to your vcenter with linked mode if you have


New-Item -Name noportgroup_vdswitch -ItemType Directory
Get-VDSwitch | % {Export-VDSwitch -VDSwitch $_ -Destination ".\noportgroup_vdswitch\NoPort_$($_.datacenter)_$($_.name)_$($_.id).zip" -WithoutPortGroups }
Compress-Archive -Path .\noportgroup_vdswitch -DestinationPath .\noportgroup_vdswitch



New-Item -Name fullportgroup_vdswitch -ItemType Directory
Get-VDSwitch | % {Export-VDSwitch -VDSwitch $_ -Destination ".\fullportgroup_vdswitch\fullport_$($_.datacenter)_$($_.name)_$($_.id).zip" }
Compress-Archive -Path .\fullportgroup_vdswitch -DestinationPath .\fullportgroup_vdswitch

New-Item -Name vdportgroup -ItemType Directory
Get-VDPortgroup | % { Export-VDPortGroup -VDPortGroup $_ -Destination ".\vdportgroup\PG_$($_.datacenter)_$($_.vdswitch)_$($_.name)_$($_.id).zip" }
Compress-Archive -Path .\vdportgroup -DestinationPath .\vdportgroup


Compress-Archive -LiteralPath .\vdportgroup.zip,.\fullportgroup_vdswitch.zip,.\noportgroup_vdswitch.zip -DestinationPath .\vdswitch_backup_$($(date).tostring("yyyyMMdd"))
 
Leave a comment

Posted by on June 26, 2020 in powercli, vmware

 

Tags: ,

PS: Get cluster CPU/Memory over-subscription

I am not the originator of these code (sorry I don’t remember where in google land they came from), but am posting it here for reference, since I may need to reuse it someday.

Note: CPU calculation is divided by 2 due to hyper-threading

#get cluster CPU over-subscription
Get-Cluster | Sort name | Select Name, @{N="CpuOversubscriptionFactor";E={[math]::Round((($_|get-VM|measure numcpu -sum).Sum)/(($_|get-vmhost|measure numcpu -sum).sum)/2,2)}}

#get cluster Memory over-subscription
Get-Cluster | Sort name | Select Name, @{N="MemOversubscriptionFactor";E={[math]::Round((($_|get-VM|measure MemoryGB-sum).Sum)/(($_|get-vmhost|measure MemoryTotalGB -sum).sum),2)}}

#get VMHost CPU over-subscription in a cluster
Get-Cluster <name> | Get-VMHost | Sort name | Select Name, @{N="CpuOversubscriptionFactor";E={[math]::Round((($_|get-VM|measure numcpu -sum).Sum)/(($_|measure numcpu -sum).sum)/2,2)}}
 
Leave a comment

Posted by on May 30, 2020 in powershell