SysAdmin Tips

Copy/Paste and Run Multi-line Bash/Shell Scripts

BASH TIP: You can copy/paste multi-line bash/shell scripts directly from sites like this and run them without syntax errors.
Type ( and then paste the contents of your clipboard, and then type ) and press enter.

Chrome Network Internals

Visit Chrome Network Internals and Cache Information

Get Public IP Address from Command Line

On a Linux terminal (or Windows with gnuwin32), you can return your public IP address with curl on ipconfig.io (see examples below).
disclaimer: I own and operate ipconfig.io

root@nginx02:~# curl ipconfig.io
47.100.16.27

root@nginx02:~# curl ipconfig.io/json
{"ip":"47.100.16.27","ip_decimal":795086875,"country":"United States","city":"Fremont"}

Weather Report in your Terminal
Run This Single Command In Terminal To Get Geeky Weather Reports
$ curl -4 http://wttr.in

WordPress NGINX Permalink Rewrite

If you’re like me and swapped out your backend Apache web server for NGINX running WordPress and changed your permalink structure from domain.com/year/mo/day/post to domain.com/post then you would have likely modified your .htaccess file to do a 301 redirect. As you know NGINX does not use .htaccess files and instead relies on the config inside your virtual server config. Luckily I have found a one-liner solution which you should insert somewhere inside your server {} bracket:

rewrite "/([0-9]{4})/([0-9]{2})/([0-9]{2})/(.*)" $scheme://$server_name/$4 permanent;

Remove all Directories in the Current Directory

If you have ever wanted to remove all directories in the current directory, it can be accomplished all with the find command. If you’re ever not sure what directory you’re in, type pwd

$ find . -type d -exec rm -r "{}" \;


WordPress FileSystem Permissions

This is a list of commands I have used in order to change the permissions on a WordPress directory structure. It is best practice to not allow the user account running the webserver software (Apache, Nginx, etc) write access to any part of the directory including the group membership, however I have always found temporary logical reasons why this would be necessary:

$ chown www-data:www-data  -R * # Let Apache be owner
$ find . -type d -exec chmod 755 {} \;  # Change directory permissions rwxr-xr-x
$ find . -type f -exec chmod 644 {} \;  # Change file permissions rw-r--r--
$ find /var/www/sysinfo.io -type f -exec chmod 644 {} \;
$ find /var/www/sysinfo.io -type d -exec chmod 755 {} \;
$ find /var/www/sysinfo.io/wp-content/ -type d -exec chmod 775 {} \;
$ find /var/www/sysinfo.io/wp-content/ -type f -exec chmod 664 {} \;
$ find /var/www/sysinfo.io -type d -exec chmod g+s {} \;
$ find /var/www/sysinfo.io/wp-content/uploads -type d -exec chmod 775 {} \;

Easily Correct a Typo of Previous Command Using Carat ^ Symbol

I know you already know you can hit the up and down arrows to navigate the command history and edit the typo, but there is one more somewhat useful trick to add to your collection of useless knowledge. Suppose you were trying to update your repo and upgrade your programs but miss-typed apt-get as ap-get like this:

root@ubuntu1604-1:~# ap-get update && apt-get upgrade -y
 No command 'ap-get' found, did you mean:
 Command 'apt-get' from package 'apt' (main)
 ap-get: command not found

Commands are easily replaced with the proper command by wrapping the typo with two carats (^) followed by the correct command which will then re-execute the entire previous line.

root@ubuntu1604-1:~# ^ap-get^apt-get

Here is the result:

 apt-get update && apt-get upgrade -y
 Hit:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease
 Get:2 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB]
 Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
 Hit:4 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease
 Get:5 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [380 kB]
 Get:6 http://us.archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages [375 kB]
 Get:7 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [321 kB]
 Get:8 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe i386 Packages [317 kB]
 Get:9 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [110 kB]
 Get:10 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [24.9 kB]

Access AWS S3 from Windows with S3 Browser

S3 Browser is a freeware Windows client for Amazon S3 and Amazon CloudFront. Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. Amazon CloudFront is a content delivery network (CDN). It can be used to deliver your files using a global network of edge locations.

Direct download

SmarTTY – a multi-tabbed SSH client with SCP support for Windows. Time to put Putty to bed.

After many years of being used to Putty on Windows, I now find myself using SmarTTY as a direct replacement because it offers additional functionality such as tabbed shells and SCP file transfer.

Windows 10 Windows Update Powershell Wrapper Script

Since the GUI for Windows Update stopped working for me I decided to write a wrapper for an existing PowerShell script to update Windows for me via Task Scheduler. Click here to download the Windows Update PowerShell Module and click here for my simple 2-line wrapper:

$winupdfile = 'Windows-Update-' + $(get-date -f MM-dd-yyyy) + '.txt'
if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator")) { Start-Process powershell.exe "-NoProfile -ExecutionPolicy Bypass -Command "Get-WUInstall -AcceptAll | Out-File $env:USERPROFILE\$winupdfile -Append"" -Verb RunAs; exit } else { Start-Process powershell.exe "-NoProfile -ExecutionPolicy Bypass -Command "Get-WUInstall -AcceptAll | Out-File $env:USERPROFILE\$winupdfile -Append""; exit }

Windows Update PowerShell Module

The PSWindowsUpdate module allow you to manage Windows Update on your computer. Module contain set of function to check, download and install or remove update from local machine. Especially useful on machines in core edition like Windows Server 2008 R2. I have also verified this works on Windows 10 Enterprise Edition.

Attachments: PSWindowsUpdate_1.3.4.zip | PSWindowsUpdate_1.4.3.zip | PSWindowsUpdate_1.4.5.zip

Module can be installed manually by downloading Zip file and extract in two places:

%USERPROFILE%\Documents\WindowsPowerShell\Modules
%WINDIR%\System32\WindowsPowerShell\v1.0\Modules

Importing the module

PS > Import-Module PSWindowsUpdate

Prerequisites:

Windows PowerShell 2.0
PSWindowsUpdate

Depending on the security policy on machine, module may need administrator permissions to use all functions.
Module can use Windows Update offline scan file (I excluded wsusscan.cab file from the module so now need to download it individually). The latest Windows Update offline scan file (wsusscn2.cab) is available from Microsoft Baseline Security Analyzer (MSBA) or System Management Server Inventory Tool for Microsoft Updates (SMS ITMU).

Available functions:

Add-WUOfflineSync
 Add-WUServiceManager
 Get-WUHistory
 Get-WUInstall
 Get-WUInstallerStatus
 Get-WUList
 Hide-WUUpdate
 Invoke-WUInstall
 Get-WURebootStatus
 Get-WUServiceManager
 Get-WUUninstall
 Remove-WUOfflineSync
 Remove-WUServiceManager
 Update-WUModule

Bypass HSTS on broken website SSL certificate

Anyone who has played around with HTTP Strict Transport Security knows that it is not an easy thing to reverse if you want to test some function while the website has a broken or expired certificate. Little trick I have learned: while on the error page in Google Chrome, type “badidea” without quotes and you will be allowed to proceed ^_^

AWS EC2 VPC Network Address Resolution

After contacting the free AWS Support channel over multiple days and trying to confirm if it was true that they transparently convert destined (outbound) public IP traffic to the corresponding internal IP address of the receiving instance, as long as both instances reside within the same AZ and within the same region. That is logical since the traffic wouldn’t go out to the internet and come right back if they used a special translation system. But apparently I have confirmation that this is not true according to AWS Support which you can see below my original tip. If you definitively know whether or not this is true please contact me.[/ct_message]
If you communicate to an running Amazon instance via its public Amazon DNS name (ec2-xx-xxx-xxx-xx.compute-1.amazonaws.com) from another instance within the same region over TCP/IP, network traffic will silently resolve and talk via its internal address as opposed to its public, which lowers latency since the two systems do not have to touch the internet, as well as save you money because availability zone to availability zone (intra-region) traffic is billed at a cheaper rate than external traffic. 

The most comprehensive guide to AWS can be found on this page here

Correspondence
Amazon Web Services
Dec 27, 2016
04:33 PM -0800
Hello,

We are glad to assist and address your concern on this matter.

If you have any technical questions in the future, we have a number of Premium Support plans that allow you to speak directly with a Support Engineer. With Premium Support, you can speak to an engineer by email, chat, or phone depending on what support plan you choose.

These plans offer a tailored support experience that allows you select the support level that best fits your needs. Premium Support plans which all include direct access to Support Engineers start out at just $29/mo. The plans are prorated, and can be cancelled any time. More information including pricing and how to sign up can be found here:

https://aws.amazon.com/premiumsupport/pricing/

Please feel free to contact us if you have any questions.

Best regards,

Suzie Q.
Amazon Web Services

Check out the AWS Support Knowledge Center, a knowledge base of articles and videos that answer customer questions about AWS services: https://aws.amazon.com/premiumsupport/knowledge-center/?icmpid=support_email_category

We value your feedback. Please rate my response using the link below.
===================================================

Was this response helpful? Click here to rate:
travis
Dec 27, 2016
04:21 PM -0800
Ok that answers my question. Amazon does not automatically translate public IP’s to private in an efficient manner. My original understanding was that given that scenario, it would automatically be translated behind the scenes so routing and latency would be reduced.

Amazon Web Services
Dec 27, 2016
04:04 PM -0800
Hello,

The instances do not require public IPs to talk to each other when they’re in the same VPC. Doesn’t matter which AZs they are in. They’re still under the same private cloud. You just need to allow private IPs of each other in the Security Group rules

Ideally, in this case IGW or the public IP addresses would not come in to the picture if these instances were to talk to each other (Even if both the instances have public IP addresses)
– Local route would keep the traffic within the VPC (10.x.x.x/x to local)
– Default route would be to IGW (0.0.0.0/0 to IGW)

If you actually want these two instances to talk each other over public IP address, then it would still use the same IGW. IGW would take care of the routing based on the public IP the traffic is destined to.

At no point would the traffic be converted from public Ip to private.

If you could provide a use case for having the two VMs in the same VPC to talk over public IP, maybe we can better address your concern. Because it is unnecessary to have 2 instances talk to each other via public IP, and would cost you for sending the traffic out of the VPC.

Below is an AWS Documentation on VPC Route Tables that may be helpful:
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Route_Tables.html

I hope this helps, please feel free to contact us if you have any questions.

Best regards,

Suzie Q.
Amazon Web Services

Check out the AWS Support Knowledge Center, a knowledge base of articles and videos that answer customer questions about AWS services: https://aws.amazon.com/premiumsupport/knowledge-center/?icmpid=support_email_category

We value your feedback. Please rate my response using the link below.
===================================================

Was this response helpful? Click here to rate:
travis
Dec 27, 2016
03:00 PM -0800
Sorry my question may have been overly complex for a simple question. Given the scenario where you have two VM’s in the same AZ in the same VPC, both have Public IP addresses. If they were to talk to each other via the public IP address, does the traffic transparently get converted to its internal IP so as to not touch the internet and save on bandwidth? Please see my horrible diagram attachment.

Attachments:
Capture.PNG
Amazon Web Services
Dec 27, 2016
02:19 PM -0800
Hello,

Sorry for the concern the traffic fees may have caused. The complexity of your question is a bit outside the billing and accounts scope. I was able to get some information from our Service Team that may answer your question.

It’s not necessary to need an EIP/Public IP for two instances in different AZs to talk to each other as long as they are in the same VPC. You don’t get charged for traffic that doesn’t traverse an IGW. And two VMs in same AZ (Infact same VPC) do not require public IP to communicate. But if the 2 VMs in the same AZ belong to different VPC, then yes, you would need a public IP and an IGW to communicate. In this case, you would get charged for the traffic.

I also find an AWS documentation on VPC that may help as well:

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Introduction.html

I hope this helps, please feel free to contact us if you have any questions.

Best regards,

Suzie Q.
Amazon Web Services

Check out the AWS Support Knowledge Center, a knowledge base of articles and videos that answer customer questions about AWS services: https://aws.amazon.com/premiumsupport/knowledge-center/?icmpid=support_email_category

We value your feedback. Please rate my response using the link below.
===================================================

Was this response helpful? Click here to rate:
travis
Dec 26, 2016
04:51 PM -0800
So my understanding that AWS’s network silently converts Public IP’s and EIP’s within the same AZ to its corresponding internal IP is false and does in fact touch the internet and back? Ie: two VM’s within the same AZ communicate with each other via their public IP will incur regional data transfer rates as opposed to intra-regional data transfer rates?

Amazon Web Services
Dec 26, 2016
01:19 AM -0800
Hello there,

My name is Zaahir from the AWS Billing and Accounts Department. I’ll be happy to assist you with this.

I’m sorry for the unexpected charge you recently incurred. The Regional Data Transfer charge of $0.98 is not included in your Data Transfer limit for the Free Usage Tier offer and is incurred when you either:

1. Transfer data in/out between instances in different Availability Zones in the same region. (example: from us-east-1a to us-east-1b)
2. Use an Elastic IP or public IP to access your instance
3. Use Elastic Load Balancing to communicate inside of the Amazon EC2 network.

In case 2 and 3, you’ll pay Regional Data Transfer rates even if the instances are in the same Availability Zone. For data transfer within the same Availability Zone, you can avoid this charge by using your private IP whenever possible.

The traffic you’re seeing may have originated from another customer in another region, causing the inbound traffic, and your instances will have responded with response packets, resulting in outgoing data transfer. It doesn’t imply that your instance initiated this traffic with another region; rather, your instance is responding to requests coming to it from another region. The first 1 GB of data transfer out of AWS for each region is free, which covers the majority of these simple requests.

You pay one rate for all combined data transfer out of AWS (except for CloudFront), up to the first 10 TB, and pricing continues to drop the more bandwidth you use. You can see our Data Transfer pricing here:

https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer

If you would like to prevent your instance from responding to these background requests, you will want to make sure your security groups are configured to block unsolicited inbound traffic to prevent unauthorized users from accessing data on your instances. Here are some tips for securing your instances against unwanted traffic:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html

I hope this helps, but please let me know if you have any further questions.

Have a lovely day further!

Best regards,

Mogamat Z.
Amazon Web Services

Check out the AWS Support Knowledge Center, a knowledge base of articles and videos that answer customer questions about AWS services: https://aws.amazon.com/premiumsupport/knowledge-center/?icmpid=support_email_category

We value your feedback. Please rate my response using the link below.
===================================================

Was this response helpful? Click here to rate:
travis
Dec 23, 2016
02:08 PM -0800
From my understanding, when traffic is destined for a public IP/EIP or public DNS address of another instance in a different AZ but in the same region, it would be charged at the rate of AZ to AZ internal IP, since the IP is translated to an internal address and does not touch the internet. If I’m incorrect please let me know. This is documented as #4 on this image: https://sysinfo.io/wp-content/uploads/2016/12/aws-data-transfer-costs.png

Attachments:
aws-data-transfer-costs.png


Download text version
Share
Disqus Comments Loading...