What we want to do

When I wanted to see the “https://learning-english.algo-ai.work/category/grammar/index.html” file, I wanted to see the “https://learning-english.algo-ai.work/category/ I wanted “grammar/” to automatically look at the index.html file. since WordPress places Index.html in each permalink directory by default, I didn’t want to do it by stating index.html after I made it static, so I looked for a way I searched for a way to do this.

Redirect by function

CloudFront allows you to select a function in addition to a distribution. We decided to write a program that will automatically write index.html at will by selecting a function from the tab.

Proceed with the creation of the function and include the following code in the development tab

function handler(event) {
    var request = event.request
    var uri = request.uri;
    if (uri.endsWith('/')) {
        request.uri += 'index.html';
    } else if (!uri.includes('.')) {
        request.uri += '/index.html';
    }
    return request;
}

Click on Publish Function and Add Associations and select the distribution to which you wish to automatically grant index.html.

This is the end of the configuration, so when you access the “Distribution Domain Name”, you will automatically be able to access the site that requires index.html.

Conclusion

We have introduced some of the problems we encountered when publishing a web server with Cloud Front, and how to make WordPress static and publish websites with Cloud Front + S3 for blazing speed. Please read it as well if you like.

Follow us on Twitter if you like.

Installing staatic

Static is a plugin that turns WordPress into a staticized site. There are multiple candidates for a staticization plugin, but I am confident that this plugin is the only one that has been successfully staticized.

Install Static from “Add Plugin” in WordPress. You can install it by searching and pressing the Install button. Press the Enable button to activate it.

A tab will appear at the bottom of the menu.

We have migrated to a different domain; the URL describes the domain you want to use.

Click on “Publish now” from Publications to make it static.

Unfortunately, however, xfree did not work in a https environment.

Where I got hooked.

In xfree, it did not work well with https, so changing the configuration to http worked.

Disable SSL settings and press Publish Now and it works.

Other important permalink settings

Some permalink settings may not work properly.
From the Settings tab, click on Permalinks to set the format of the permalinks that are supported.

Myself, I selected “post name” and it worked. The folder structure is such that index.html is placed in a folder with the name of the article, such as /sample-post/.

In addition, when going to SSL, I was stuck with the fact that it would not access index.html for each folder; Cloud Front uses Function to redirect it, and it works. I will show you how to do this in another article.

Conclusion

I have shown you how to staticize your WordPress pages with Static, I thought it would be cosmetic to publish them with S3 + Cloud Front, so I am staticizing them and publishing them on the web. I like the fact that it eliminates the need for specs on the web server where I write the articles.

Follow us on Twitter if you like.

M1 Get a Let’s Encrypt certificate on your Mac

Get a free certificate at Let’s Encrypt.
M1 We will show you how to obtain a certificate on a Mac environment, assuming that you are able to publish your webbase on S3. The following article will show you how to do this.

Install certbot.

brew install certbot

After installation, it is installed in [/opt/homebrew/Cellar/certbot/2.1.0/bin/certbot].
In my environment, the link was not put in bin. I tried to use it as it is without linking.

/opt/homebrew/Cellar/certbot/2.1.0/bin/certbot --version
certbot 2.1.0

command seems to work. Let’s create the certificate as is without the symbolic link.
If you want to issue a certificate for learning-english.algo-ai.work, the command will look like this

/opt/homebrew/Cellar/certbot/2.1.0/bin/certbot certonly --manual -d learning-english.algo-ai.work

When you type, the message “Create a file containing just this data” is displayed, so create a file in S3 for authentication, place it there, and authenticate it.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Create a file containing just this data:

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

And make it available on your web server at this URL:

http://learning-english.com/.well-known/acme-challenge/xxxxxxxxxxxxxxxxxxxxxxxxxxxx

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

Upload the entire .well-known to S3 and press [Enter] to approve and issue the certificate.

The set of certificates will be located in “/etc/letsencrypt/live/learning-english.algo-ai.work/”.

Send a set of certificates to Cloud Front

Send a set of certificates to Cloud Front via AWS command.

Create IAM user

First, create a policy to grant privileges to the IAM user: create the following in the JSON tab, return to the visual editor, continue to the next step and press the “Create Policy” button. If you do not see the button on the screen, you can see it by reducing the text on the web screen.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "iam:*ServerCertificate",
                "iam:*ServerCertificates"
            ],
            "Resource": "*"
        }
    ]
}

After creating the policy, open IAM and create a user.

Add a user.

Create a user with certificate privileges in the policy you created.

Select the user you created and choose the Authentication Information tab.

In the “Access Key” field, you will create an access key.

You will get an access key and secret access, which you should write down and use later in aws console.

Send certificate to Cloud Front with aws cli

Install the awscli command on m1 mac.

brew install awscli

Type “aws configure” and you will be asked for the Access Key ID and Secret Access Key.

aws configure
AWS Access Key ID [****************]: 
AWS Secret Access Key [****************]: 
Default region name [ap-northeast-1]: 
Default output format [json]: 

Upload the certificate using the aws command. Change the directory name to your domain name and execute.

aws iam upload-server-certificate --server-certificate-name learning-english.algo-ai.work-ssl 
--certificate-body file:///etc/letsencrypt/live/test.algo-ai.work/cert.pem 
--private-key file:///etc/letsencrypt/live/test.algo-ai.work/privkey.pem 
--certificate-chain file:///etc/letsencrypt/live/test.algo-ai.work/chain.pem 
--path /cloudfront/

Set up certificates in Cloud Front

Set the created certificate to Cloud Front.
Create a distribution and select the origin domain where the published web server information is located.

Set an alternate domain name (CNAME) to use your own domain.

Select the SSL certificate. If you have the certificate you just uploaded, you have succeeded.

Cloud Front should be able to publish the web server. If you can wait a few minutes, copy the distribution domain name and access it with the URL in another tab of your browser, you have succeeded.

Connect your own domain to CloudFront with Name.com

Set up a cname at Name.com so that you can access the S3 URL from your own domain.

Configure CNAME settings in DNS.

CNAME records are a mechanism that allows you to associate an alias with a canonical name, so that you can access AWS with your FQDN, and then associate it with your canonical name to access your own domain.

Go to Name.com and from the Domains tab, click on “Domain DNS Settings”.

Click the “Configure” button under Use DNS record settings.

Add a record, selecting “CNAME” for TYPE, with any subdomain name for the hostname and the AWS CloudFront URL for the VALUE. Click the ADD button to confirm and configure.

Conclusion

We have shown you how to use Let’s Encrypt certificates to SSL-ize your website on AWS Cloud Front. Now you can build a CDN environment with enhanced security, and create a blazing-fast website.

Follow me on Twitter if you don’t mind

How to Auto Post on Instagram

The way to automatically post on Instagram is to automate scheduled posts with RPA. Specifically, I use Statusbrew to schedule posts. For free, the maximum number of reserved posts is 10, and reposts must be manually registered. Automation is performed with the Chrome extension UI Vision RPA, and automatic booking is performed by periodically executing the operation with the Chrome extension crontab. After booking, you can modify the content of the post by editing, or you can post the draft as it is.

We also introduce how to auto-like and auto-follow, so please read it if you like.Chrome extension that automatically follows and likes InstagramIntroducing the Instagram automatic follow / automatic like function that can be used in Chrome! Easily increase your followers and likes … https://algo-ai.work/blog/2023/01/14/auto-instagram/

Advance preparation

Instagram or Facebook must be a professional or business account. Also, it is necessary to link Instagram and Facebook in advance.

Switch Instagram to a professional account.

Click Edit Profile.

Click Switch to Pro Account to switch your account to a Pro Account.

Also, link Instagram and Facebook, and configure settings to allow linking from Instagram to Facebook and settings to allow linking from Facebook to Instagram.

From Facebook, there is an item called Instagram in the settings, and you can link it by clicking “Link Account”.

From Instagram, you can link with Facebook by clicking “Login” from the item “Linked accounts” in the settings.

In addition, Facebook also requires a business account, so click Create from the “Page” item to create a page. If you do not create a page, it will be flipped when linking with Statusbrew.

Statusbrew

Statusbrew unifies content marketing management for social media, risk management measures, engagement management including comment monitoring, optimizes internal approval workflows, and enables high-performance analysis. In the free version, you can schedule posts, define categories, define plans, etc. In addition, you can easily post one post to multiple SNS such as Twitter, Facebook, and Instagram. This time, I will use it for the purpose of posting on Instagram.

Introducing Statusbrew

Create a user and log in to create a space. Enter any name and click “Create”.

You can connect social profiles by creating a space and selecting the created space.
Click Instagram.

I clicked on “Login via Facebook”.

Allow Statusbrew from the app and website in your Instagram settings.

It will be a success if we can cooperate safely.

How to use Statusbrew

I thought Statusbrew would take some getting used to.

You can access the top page by logging in and pressing the logo on the top left. In the free version, you can set calendar and category posts.

You can schedule posts on the calendar, and you can set a schedule pattern to use on the calendar for category posts. If you use this category post well, you can efficiently set up reservations.

Category post

For category posts, you can assign a category name and create a reservation pattern.

For example, if you set it as follows, it is a pattern that can be set so that the same post can be made at 6:45 in the morning on Monday, Wednesday, and Friday. It can be patterned and, if used well, can be efficient.

It is convenient when making a reservation if you use a name that is easy to understand.

Post draft

Create drafts instead of direct bookings for automation. I’m ready to duplicate the draft and schedule it.

Click the paper plane icon on the left and select Drafts.

Click Create Post.

Enter your social profile, text, and upload the photos you want to subscribe to on Instagram. Since it is a regular reservation, you can replace the photo after making a reservation. Check “Add Draft”.

In Category Reserve, select the category you created. It’s nice because you can make a pattern reservation instead of the date and time.

Click the Add Draft button to create a draft.
In preparation, you will create multiple drafts of this. Up to 10 items can be reserved at once in the free version. With the method introduced in this article, you can make a free and flexible fully automatic reservation by creating a category for each month to day. I’m going to be so detailed, so I tried to create it all together for the time being. I thought that it would be enough to make an automatic reservation about twice a week.

Install Chrome extension

For automation, we use the Chrome extension UI Vision RPA and crontab. It is an RPA tool that can automate Chrome operations and a tool that can be regularly executed with pre-scheduled content.

Scheduled execution with Chrome extension

Duplicate drafts in UI Vision RPA for automatic booking.

Open UI Vision RPA and click the “+Macro” button to create a new macro.

Click the “Record” button to let Statusbrew automate and record Chrome activity.

Specifically, on the draft screen, click the draft you created, click “…”, and select “Duplicate Post”.

When the screen changes, perform the operations up to posting and have RPA remember it. Uncheck “Add Draft”, select “No Approval” in the “Assign Approver” item, and click “Reserve Post” to complete. It won’t be posted right away, so press Scheduled Post to test it. When done, press the UI Vision RPA’s “Stop” button to end the Record.

I was able to record the operation with UI Vision RPA. Run “Play Macro” to see if it works. I have a macro that can repost if it works properly.

Convert macro to HTML

UI Vision RPA can be downloaded in HTML format.

Right-click the macro you created and select Create autorun HTML.

You can download the html file by selecting it. From now on, you can run the recorded macro simply by opening this HTML file.

Periodic execution

Regular execution is executed with crontab of chrome extension. Up to this point, just by opening the HTML file, you can duplicate the draft and schedule posting.

Open Chrome’s crontab and schedule it for regular execution.

Use the URL displayed in chrome when executing the html downloaded with the RPA tool. Add “?direct=1” and set it. In the following case, we are scheduling to directly open the RPA html at 00:00 on Sunday. The option is set to open directly with “?direct=1”. It doesn’t work without settings. Caution is required.

With this, you can regularly schedule posts on Mondays, Wednesdays, and Fridays at 00:00 every Sunday. Since the categories were grouped together, the schedule was rough, but if you set it in detail, you can automate the reposting of different images every day. By creating RPA, it may be possible to replace images.

Conclusion

I introduced how to automatically post by making an automatic reservation on Instagram for free. Auto-posting on Instagram is considered bad, but I don’t think there’s a problem with auto-booking. I think it would be a good idea to make an automatic reservation, basically change the image if you have time, and automatically repost the image that was viewed most often in the past when you don’t have time. .

Please follow me on Twitter if you don’t mind .

Chrome extension

The Chrome extensions I use are:
It is a method to automatically run like & follow by opening the html file.

• Superpowers for Instagram: Auto-follow and auto-like 20 users with one click
• UI Vision RPA: Automate browser operations with RPA
• cronTab: allows you to run commands periodically in your browser

First, install three Chrome extensions.

Even if you don’t use “Superpowers for Instagram”, you can operate UI Vision RPA to make it like and follow, but it feels like it can be used as a module for more stable operation. , I decided to use it this time. It can slightly compensate for unstable operation, which is a drawback of RPA.

Superpowers for Instagram

Try Superpowers for Instagram.
Open Instagram and open the page of the tag you want to follow or like.

As a test, open “https://www.instagram.com/explore/tags/followforfollowback/” and you’ll find a “Like all” button and a “Follow all” button. By clicking this, you can like 20 people or follow 20 people for the post of the person who posted most recently.

UI Vision RPA

UI Vision RPA can record browser operations and execute recorded records with Play Macro.
Press Record and press any of the “Like all” or “Follow all” buttons in “Superpowers for Instagram” to memorize your browser actions. It can be stored in JSON format.
From the next time onwards, you can execute it by pressing Play Macro. After memorizing it, check that it can be executed. When I tried it myself, I was able to get the following JSON code. You can also write this code in JSON of UI Vision RPA and execute it.

{
  "Name": "instagram",
  "CreationDate": "2023-1-14",
  "Commands": [
    {
      "Command": "open",
      "Target": "https://www.instagram.com/explore/tags/followforfollowback/",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "waitForElementVisible",
      "Target": "xpath=/html/body/aside/div[3]",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "click",
      "Target": "xpath=/html/body/aside/div[3]",
      "Value": "",
      "Targets": [
        "xpath=/html/body/aside/div[3]",
        "xpath=//aside/div[3]",
        "css=body > aside > div.sfi-button.sfi-button--like"
      ],
      "Description": ""
    }
  ]
}

JSONCopy

Parameters such as timeouts and command execution intervals can be set by pressing the gear settings button in UI Vision RPA. If a timeout occurs and it doesn’t work, you should increase the timeout interval or the command execution interval. Although it becomes an unstable operation peculiar to RPA, I think that it is enough if it works.

Left click on the project you are creating and select “Create autorun HTML”. You can download the HTML file. When you double-click this HTML, it will automatically like and follow. After that, if you can open this file regularly, you will be able to do regular automatic likes and follows.

cronTab

Use cronTab to open this HTML file periodically. An example of running once an hour is when Minute is divisible by 40. You can run it directly by setting direct=1. If you forget to add the option, it will not work properly. Caution is required.

With this, the html file will be opened every hour and it will be a success if you can automatically follow and like automatically.

Conclusion

In this article, we introduced how to use the Chrome extension to automatically follow and like Instagram on a regular basis. You can achieve Insta automation with the help of 3 Chrome extensions. RPA operations are difficult to stabilize, but they are versatile techniques. It can be applied to various things.
We also introduce how to periodically run Google Colaboratory, which allows you to use the GPU for free, so please read it if you like.

Please follow me on Twitter if you don’t mind .

I asked AI for a café-like song

I visited “AI’s café-like song” and asked him to give me a concrete English sentence. I asked a question using the open AI API on Slack. Chat GPT can support Japanese and speed up. The article below introduces how to use chat GPT with Slack, so please read it if you like.

Confirmed in Slack to chat GPT.

He specifically taught me slow, light music. This time, we will use this character as an input character string to generate music.

text to music 「mubert」

Text to image is in vogue, but text to music is being created. You can create music from text. With mubert, you can generate music by specifying three tag information.

Please try it.

Using the Mubert API, get_track_by_tags determines 3 tag information from a sentence. The code was published on Google Colab. I copied and pasted the characters that chat GPT output to prompt and executed it.

generate music

I entered the characters output by Chat GPT in Prompt and executed it.

The tags of the cafe-like song were the following three.

• jazz / funk
• electro funk
• sleepy ambient

I generated music from these three tags. duration is 60 seconds.

Conclusion

I made a café-like song concrete with chat GPT, converted it to 3 tag information, and generated non-existent music. As far as actually listening to music, text to music is still in its infancy, but I think it has great potential. By automating MIDI and lyric linkage, linking with NEUTORINO, etc., Vocaloid singing smoothly, etc., I think that if automation is developed, an integrated automatic music generation environment can be created. I’m looking forward to the world of automatic music generation in the future.

Please follow me on Twitter if you don’t mind .

How to use chat GPT in Line

Using OpenAI’s API, you can use Google Apps Script to support LINE to use AI that supports Japanese. You can try it out for free, and you can use it conveniently by paying out your LINE business account, writing a simple script in Google Apps Script, and linking it together to support it.
Please try the following article for more details.
In addition, please read “Translating with Google Apps Script” on this page, and you can try the highly accurate chat function by using the additional translation API.

How to use chat GPT in Slack

Using OpenAI’s API, Google Apps Script can be used to support Slack so that it can be easily touched by the team and also in Japanese.
If you can integrate it into Slack, it would be convenient for your team to touch it easily.
Please check the following article for more details.
In addition, please read Translating with Google Apps Script on this page to try out the highly accurate chat functionality by using the additional translation API.

Translate with Google Apps Script

getChatGptMessage is functionalized in the script created in the previous section.
Google Translate is interspersed with LanguageApp.translate.
Translate Japanese into English and use openAI’s API in getChatGptMessage.
Store the response in content, translate it and store it in content.
Now you can use chat GPT with high accuracy by using the API in English and also in multiple languages.

var temp;
temp = LanguageApp.translate(message,"ja","en"); //English
var content = getChatGptMessage(temp);
content = LanguageApp.translate(content,"en","ja"); //Japanese

Conclusion

We have shown you how to use chat GPT in a highly accurate way in Japanese.
To ask questions in English, you can use the translation API for better accuracy, so please try it out if you use chat GPT on a regular basis.

Make it compatible with Line

Using OpenAI’s API, you can use Google Apps Script to support LINE to receive answers to replies easily and in bulk.
It is possible to put this into practice free of charge.
If you can use it on LINE, you can easily use it as if you were Googling.
Please check the following article for more details.

Make it compatible with Slack

Using OpenAI’s API, you can use Google Apps Script to support Slack, making it easy for your team to touch and receive responses to replies in bulk.
This is a free practice.
If you can integrate it into Slack, it would be convenient for your team to touch it easily.
Please check out the following article for more details.

Conclusion

We are pleased to present an article on easy access to chat GPTs.
If you can touch the latest AI chatbot embedded in LINE or Slack, you can make some progress.
Please give it a try.

How to obtain an access token and secret token for LINE Bussiness

Register and login to LIne Bussiness.

Click Create to create a new BOT.
Agree to create an account and create a provider.

Click on Chat to configure Webhook settings.

Click on the Chat tab, then click on “Open Messaging API Settings Window.

The Webhook URL will contain the URL of the server that you later scripted and Deployed in GAS.
When this LINE account receives a message, it can send the message received to this server URL to the doPost function.

Click on the Line Developpers link.

Click on the console, select the provider you created, and click on the Messaging API Settings tab.

Issue a channel access token.
This access token will be used in the script, so keep it in mind.
This will be used when replying (replying to a message) to LINE from the server side.
Also, in the response settings, turn off chat and turn on Webhook.

How to obtain an API key for OpenAI

Obtain an OpenAI API key and prepare to create a server-side script.

Sign up below to login, you can also register with a Google Account.

Log in and click Create New Seacret Key to obtain a key for the API.
You are now ready to create your script.

Google Apps Script (GAS) Script

Prepare a server in a serverless environment with Google Apps Script.
Create a project and script at the following URL

In the script, enter the string that you have just memorized in the [your OpenAI API key] and [your Line channel access token] fields.

// Call ChatGPT's API and return response
function getChatGptMessage(message) {
  var uri = 'https://api.openai.com/v1/completions';

  var headers = {
    'Authorization': 'Bearer [API key of own OpenAI]',
    'Content-type': 'application/json'
  };

  var options = {
    'muteHttpExceptions' : true,
    'headers': headers, 
    'method': 'POST',
    'payload': JSON.stringify({
      "model": "text-davinci-003",
      "max_tokens" : 2048,
      "prompt": message})
  };
  try {
      const response = UrlFetchApp.fetch(uri, options);
      var json=JSON.parse(response.getContentText());
      return json["choices"][0]["text"];
  } catch(e) {
    console.log('error');
  }
}
//Get e object when message is received
function doPost(e) {
  let token = "[Own Line channel access token]";
  // Obtained as JSON data
  let eventData = JSON.parse(e.postData.contents).events[0];
  // Obtain tokens for reply from JSON data
  let replyToken = eventData.replyToken;
  // Get message from JSON data
  let userMessage = eventData.message.text;
  // Define API URL for response messages
  let url = 'https://api.line.me/v2/bot/message/reply';
  // Prepare reply message from JSON message
  let replyMessage = userMessage;

  //Set payload value with text for messages returned by the defined chat GPT
  let payload = {
    'replyToken': replyToken,
    'messages': [{
        'type': 'text',
        'text': getChatGptMessage(replyMessage)
      }]
  };
  //Set POST parameter for HTTPS
  let options = {
    'payload' : JSON.stringify(payload),
    'myamethod'  : 'POST',
    'headers' : {"Authorization" : "Bearer " + token},
    'contentType' : 'application/json'
  };
  //Request and reply to LINE Messaging API
  UrlFetchApp.fetch(url, options);
}

Once you have created the script, click New Deploy, click the gear and select Web app. in Who has, select anyone and click Deploy. log in with Authorized access and approve.

Click Deploy to create a web URL and copy it.

Paste it into the LINE Webhook URL, save it, and you are done.

You can now freely use the chat GPT, on line.

Conclusion

We have shown you how to easily touch the chat GPT on line.
Now you can consult chatBot anytime. If you have the ability to use AI on your smartphone, you can write enough code. It is also great that you can easily use it as if you were Googling.

We believe that everyone is implementing test automation.
If you want to automate a combined or comprehensive test that straddles multiple devices rather than a single device, you can do so flexibly with shell scripts, which we will introduce here.

#!/bin/bash
sudo ssh -i ~/.ssh/id_rsa -o 'StrictHostKeyChecking no' -t user@x.x.x.x

% sh login.sh

#!/bin/bash
date >> test_work.txt
date
exit

Execute shell script

Prepare a work.sh script that describes what you want to execute. Pass this script file at login.
After multiple shell executions, download the file that outputs the execution results and receive the results.

◆auto.sh

sh login.sh 'sh' < work.sh
sudo scp -P 22 -i ~/.ssh/id_rsa user@x.x.x.x:/home/user/test_work.txt ./

When executed, the file can be retrieved automatically.

sh auto.sh

You can also run multiple shells simultaneously.
By running them in the background (&) and waiting for the process to finish, you can make simultaneous connections via ssh.

◆auto2.sh

sh login.sh 'sh' < work.sh &
sh login2.sh 'sh' < work.sh &
wait
sudo scp -P 22 -i ~/.ssh/id_rsa user@x.x.x.x:/home/user/test_work.txt ./
sudo scp -P 22 -i ~/.ssh/id_rsa user@y.y.y.y:/home/user/test_work.txt ./

Execute shell scripts.

sh auto2.sh

Logging by Script

Here is an example of how to apply this to access and log the web while retrieving log files.
The following script will retrieve a new Nginx access log for 10 seconds.

◆work2.sh

(sleep 10 ;kill $$) &
exec sudo tail -n 0 /var/log/nginx/access.log > ~/access.log

◆auto3.sh

While accessing with work2.sh, you can automate the sequence of accessing from your own terminal with curl and acquiring logs, so that you can perform the test.

sh login.sh 'sh' < work2.sh &
curl http://x.x.x.x
wait
sudo scp -P 22 -i ~/.ssh/id_rsa user@x.x.x.x:/home/user/access.log ./

This can be done as follows.

sh auto3.sh

Examination of results

If the end result is to compare the results with the expected values, I believe the test can be automated by compiling the downloaded log files into Excel and comparing them in Excel.

How to split a window (split cell display) in Google Colab.
You can split and display cells in the same way as you split and develop in Vim on Temirnal.
I think you can make good use of it when you describe a long program with cells.

How to split a Google Colab window

Click on the red frame of a cell in the image.
The cell will be displayed in a split view when clicked.

If multiple cells are used, only the same cell will be duplicated and displayed.

To display a duplicate of another cell, click the button in the same way in another cell.

Vertically split cell display

You can also change the view to vertical by clicking on the vertical line in the upper right corner.

Cells can be split vertically.

Conclusion

We have introduced a method for splitting a window (how to split a cell into separate displays).
I think this can be used effectively when describing cells in a long program.

Environment

• Cent OS 7
• Nginx 1.16.1

403 Forbidden

Create a test.html file.

# echo "Hello World" > test.html

403 Forbidden.
Depending on your environment, you may not be able to access the file even if you create a new file.

Confirmation of setting status

Check access rights.
The following commands can be used to check.

# ls -ltr --context test.html

The options are detailed below.
You will also need to check the label.

• -l : Show details in long format.
• -t : Display files in reverse chronological order.
• -r : Reverse the sort order.
• –context : Show SELinux security context for each file.

Here is an example of the result.
Check other files in the same folder to see the differences.

-rwxr-xr-x. nginx nginx unconfined_u:object_r:user_tmp_t:s0 test.html

Change of access rights

If “-rwxr-xr-x” is not what you expect, check the other files and change the permissions.
You can use the chmod command to change it.

# chmod 755 test.html

Change of Ownership

If “nginx nginx” is not what you expect, check the other files and change the ownership.
You can use the chown command to change it.

# chown nginx.nginx test.html

Label Change

If “unconfined_u:object_r:user_tmp_t:s0” is not what you expect, check the other files and change the label.
You can use the chcon command to change it.

# chcon unconfined_u:object_r:user_tmp_t:s0 test.html

Conclusion

I think it is easy to miss the point that you need to change the label.
If you are having trouble with 403 Forbidden, I suggest you give it a try.
If you are interested, please read how to create a GUI environment using the Oracle Cloud’s perpetual free framework.

I will introduce the second part on how to use EA with a free VPS. In the first part, I posted how to create a VM with OCI and make an SSH connection from your own terminal. In the second part, I will post how to enable a GUI connection to the VPS connected via ssh and start the Windows application MT4 on CentOS.

GUI for Cent OS

Make CentOS GUI. You should be able to make an SSH connection at this point, but you cannot access it via GUI; install GNOME and make it GUI.

sudo yum -y groupinstall "GNOME Desktop"
sudo systemctl start xrdp.service
chkconfig xrdp on

Since the Linux Firewall is already configured, it is also necessary to drill a hole in port 3389 on the server side. Since the VCN configuration for the cloud has already been done in the previous section, it is assumed that the drilling of the hole has already been completed.

sudo firewall-cmd --permanent --zone=public --add-port=3389/tcp
sudo firewall-cmd --reload
sudo firewall-cmd --list-all-zones

Connect via Remote Desktop

First, create an account for remote desktop connection via the CLI. Use the following command to create the account and set a password. Be sure to remember the account name and the password you set.

sudo adduser xrdp_user
sudo passwd xrdp_user

After completing the settings on the CUI, you can connect from your own terminal using remote desktop. We recommend using Microsoft Remote Desktop to connect . After installing the application, press the “+” button and “Add PC”. Enter the public IP of the OCI VM in PC Name. Set the set account and password in User Account. In addition, if the purpose is to move MT4 a lot, lower the resolution of the display and reduce the processing of the VM by RDP. On the “Display” tab, select “640 x 480” for Resolution. And if you can save and connect by double-clicking, the GUI login is successful.

Deployment of Wine to run Windows software on Linux

Even if you can log in with the GUI, you cannot start Windows applications as it is, so you will need to install Wine, a software that allows you to run Windows applications on Linux. Since it requires manual make, the process will take some time. log in with GUI, launch Terminal, and input the following reference commands at once and leave it alone. Because of the time required, it is better to enter them all at once and leave it for a day if possible.

sudo yum -y update && sudo yum install samba-winbind-clients -y && sudo yum groupinstall 'Development Tools' -y && sudo yum install glibc-devel.{i686,x86_64} gnutls-devel.{i686,x86_64} freetype-devel.{i686,x86_64} alsa-lib-devel.{i686,x86_64} -y && sudo yum install libtiff-devel.{i686,x86_64} libgcc.{i686,x86_64} libX11-devel.{i686,x86_64} libxml2-devel.{i686,x86_64} -y && sudo yum install libjpeg-turbo-devel.{i686,x86_64} libpng-devel.{i686,x86_64} libXrender-devel.{i686,x86_64} -y && sudo yum install wget && mkdir wine && cd wine && wget https://dl.winehq.org/wine/source/6.x/wine-6.6.tar.xz && tar Jxfv wine-6.6.tar.xz && cd wine-6.6 && mkdir -p wine32 wine64 && cd wine64 && ../configure --enable-win64 && make && cd ../wine32 && ../configure --with-wine64=../wine64 && make && cd ../wine32 make install && cd ../wine64 && make install 

Running Windows software. (Running MT4)

Run Windows software with the installed wine.
Run automatic trading software mt4 as a software that operates automatic trading for a long time for 24 hours. First, let’s download MT4. XM can be downloaded from here . Gemforex users can download it by clicking the platform and MT4 for PC from here .

After obtaining the exe file, copy and paste the file to the remote desktop and launch the terminal. Move to the directory where the exe file is located, and if you can execute the exe file for setup with the wine command in the installed wine folder, MT4 will start. Then install it in any folder. From the next time onwards, MT4 can be started by starting terminal.exe with the same wine command. Also, when installing multiple MT4s, it is recommended to use separate CentOS workspaces.

/home/opc/wine/wine-6.6/wine32/wine gemtrade4setup.exe

Conclusion

We have introduced a free Cent OS VM startup, RDP support, and MT4 boot support. We know it’s a bit challenging, but two VPSs for free in perpetuity is attractive.

Open a demo account with Oanda

API利用のためデモ口座を開設します。デモ口座は5分程度で簡単に作れます。

1. oandaにアクセス
2. デモ口座を開設
3. APIキーの発行

画面の赤線のところを押して、ページを移動したら分かると思います。ここで取得した APIキーは使用しますのでメモしておきましょう。

Open a demo account to use the API. You can easily create a demo account in less than 5 minutes.

1. Visit oanda
2. open a demo account
3. API key issuance

You can see it by pressing the red line on the screen and moving the page. Make a note of the API key obtained here as it will be used.

Whole Code

# coding:utf-8
# Install oandapy
!pip install git+https://github.com/oanda/oandapy.git

# import Library
import time
import oandapy
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
import pytz

# oanda API
api_key = '' # Input API Key
oanda = oandapy.API(environment = "practice", access_token = api_key)

# Function to retrieve 15 years of FX data and output a file (period and interval must be specified)
def get_histry_data(file_path, kind,duration,year_start,year_end,month_start,month_end):
    file_name =  kind + '_' + duration +'.txt'
    ys = year_start
    ye = year_start
    ms = month_start
    me = month_start + 1
    res = pd.DataFrame(None)
    first_stock = 1
    while ye < year_end or (ye == year_end and me <= month_end) :
        fmt = '%Y-%m-%dT%H:%M:00.000000Z'
        # Convert year and month data to be retrieved into strings that can be used by oandapy's api
        start1 = datetime(year=ys, month=ms, day=10,hour=12, minute=5, second=0).strftime(fmt)
        end1   = datetime(year=ys, month=ms, day=25,hour=12, minute=0, second=0).strftime(fmt)
        start2 = datetime(year=ys, month=ms, day=25,hour=12, minute=5, second=0).strftime(fmt)
        end2   = datetime(year=ye, month=me, day=10,hour=12, minute=0, second=0).strftime(fmt)

        # Data acquisition using oandapy
        res1 = oanda.get_history(instrument = kind,start = start1,end = end1,granularity = duration)
        res2 = oanda.get_history(instrument = kind,start = start2,end = end2,granularity = duration)

        # Print the time for which data is to be acquired
         #print(start1 + " " + end1)
         #print(start2 + " " + end2)

        # Convert data for one candlestick into a DataFrame
        res1 = pd.DataFrame(res1['candles'])
        res2 = pd.DataFrame(res2['candles'])

        # Data format conversion and change to Japan time
        res1['time'] = res1['time'].apply(lambda date: datetime.strptime(date, '%Y-%m-%dT%H:%M:%S.%fZ'))
        res2['time'] = res2['time'].apply(lambda date: datetime.strptime(date, '%Y-%m-%dT%H:%M:%S.%fZ'))
        res1['time'] = res1['time'].apply(lambda date: pytz.utc.localize(date).astimezone(pytz.timezone("Asia/Tokyo")))
        res2['time'] = res2['time'].apply(lambda date: pytz.utc.localize(date).astimezone(pytz.timezone("Asia/Tokyo")))
        res1['time'] = res1['time'].apply(lambda date: date.strftime('%Y/%m/%d %H:%M:%S'))
        res2['time'] = res2['time'].apply(lambda date: date.strftime('%Y/%m/%d %H:%M:%S'))

        # Repeat process for the next month
        # When the month is 13, the value is modified to be January of the next year.
        ms += 1
        me += 1
        if ys == 13:
            ys = 1
        if ye == 13:
            ye = 1
        if ms == 13:
            ms = 1
            ys += 1
        if me == 13:
            me = 1
            ye += 1

        # Combining two sets of acquired data
        res = res.append(res1)
        res = res.append(res2)

        # Export to file, but add HEADER information only the first time
        if first_stock == 1 :
            res.to_csv(file_path)
            first_stock = 0 
        else :
            res.to_csv(file_path, mode='a', header=None)
        res = pd.DataFrame(None)

#main ---------------------------------------------------------------------------------------------------------------
# Where to save files Can also be saved to GoogleDrive
path = './'

# List of Available Currencies
kind = 'USD_JPY'

# List of acquisition intervals
duration = 'M5'

# Print the path of the saved file
file_path =  path + kind + '_' + duration +'.txt'
print(file_path)

# get_data(currency_type,time_width,start_year,end_year,start_month,end_month) (get data up to 10 days)
get_histry_data(file_path,kind,duration,2005,2020,1,1)

# Load and print saved data
data = pd.read_csv(file_path)
print(data)

How to use the code

The get_histry_data function can be used to specify the type of currency and the type of time frame.

get_histry_data(file_path, kind,duration,year_start,year_end,month_start,month_end)

• file_path: file path
• kind: kind of currency
• duration: interval of acquisition currency
• year_start: start year
• year_end: end year
• month_start: start month
• month_end: end month

By changing these parameters and calling the get_histry_data function, 15 years of data can be retrieved. oanda.get_history actually retrieves prices, but only up to 5000 data can be retrieved, so one month of data is retrieved in two batches. Therefore, the program we have created this time retrieves data for one month in two parts. Therefore, the program we have created this time can only retrieve data up to 5 minutes in the shortest interval. In other words, one-minute data cannot be acquired. Also, when using get_history, we have left the code that can output the time used. if you uncomment lines 48 and 49, you can output the period of acquisition, so please try it.

Various data acquisition

The list of currencies that can be obtained with oanda’s API is open to the public. Below is a list of available currencies.

List of Currency Types

The list is below, but please check the oanda HP for details.

‘USD_JPY’,’EUR_JPY’,’AUD_JPY’,’GBP_JPY’,’NZD_JPY’,’CAD_JPY’,’CHF_JPY’,’ZAR_JPY’,        

‘EUR_USD’,’GBP_USD’,’NZD_USD’,’AUD_USD’,’USD_CHF’,’EUR_CHF’,’GBP_CHF’,’EUR_CHF’,       

‘EUR_GBP’,’AUD_NZD’,’AUD_CAD’,’AUD_CHF’,’CAD_CHF’,’EUR_AUD’,’EUR_CAD’,’EUR_DKK’,       

‘EUR_NOK’,’EUR_NZD’,’EUR_SEK’,’GBP_AUD’,’GBP_CAD’,’GBP_NZD’,’NZD_CAD’,’NZD_CHF’,      

‘USD_CAD’,’USD_DKK’,’USD_NOK’,’USD_SEK’,’AUD_HKD’,’AUD_SGD’,’CAD_HKD’,’CAD_SGD’,      

‘CHF_HKD’,’CHF_ZAR’,’EUR_CZK’,’EUR_CZK’,’EUR_HKD’,’EUR_HUF’,’EUR_HUF’,’EUR_PLN’,       

‘EUR_SGD’,’EUR_TRY’,’EUR_ZAR’,’GBP_HKD’,’GBP_PLN’,’GBP_SGD’,’GBP_ZAR’,’HKD_JPY’,       

 ‘NZD_HKD’,’NZD_SGD’,’SGD_CHF’,’SGD_HKD’,’SGD_JPY’,’TRY_JPY’,’USD_CNH’,’USD_CZK’,       

 ‘USD_HKD’,’USD_HUF’,’USD_INR’,’USD_MXN’,’USD_PLN’,’USD_SAR’,’USD_SGD’,’USD_THB’,        

‘USD_TRY’,’USD_ZAR’

List of acquisition intervals

A list of acquisition intervals is displayed.

• M: Monthly information
• W: Weekly information
• D: Daily information
• H2: 2-hour information
• H1: Hourly information
• M30: 30 minute information
• M10: 10 minute information
• M5: 5-minute information

Get all information on 78 types of exchange rates in 8 types of timeframes

It may be a little greedy, but it will be the code to get all the data for 15 years. It takes a lot of time to execute, and I think it puts a lot of load on Oanda’s servers. If you run it on GoogleColab, save your data to Google Drive so you only have to run it once.

# path = '/content/drive/My Drive/
# List of Available Currencies
kind = ['USD_JPY','EUR_JPY','AUD_JPY','GBP_JPY','NZD_JPY','CAD_JPY','CHF_JPY','ZAR_JPY',
        'EUR_USD','GBP_USD','NZD_USD','AUD_USD','USD_CHF','EUR_CHF','GBP_CHF','EUR_CHF',
        'EUR_GBP','AUD_NZD','AUD_CAD','AUD_CHF','CAD_CHF','EUR_AUD','EUR_CAD','EUR_DKK',
        'EUR_NOK','EUR_NZD','EUR_SEK','GBP_AUD','GBP_CAD','GBP_NZD','NZD_CAD','NZD_CHF',
        'USD_CAD','USD_DKK','USD_NOK','USD_SEK','AUD_HKD','AUD_SGD','CAD_HKD','CAD_SGD',
        'CHF_HKD','CHF_ZAR','EUR_CZK','EUR_CZK','EUR_HKD','EUR_HUF','EUR_HUF','EUR_PLN',
        'EUR_SGD','EUR_TRY','EUR_ZAR','GBP_HKD','GBP_PLN','GBP_SGD','GBP_ZAR','HKD_JPY',
        'NZD_HKD','NZD_SGD','SGD_CHF','SGD_HKD','SGD_JPY','TRY_JPY','USD_CNH','USD_CZK',
        'USD_HKD','USD_HUF','USD_INR','USD_MXN','USD_PLN','USD_SAR','USD_SGD','USD_THB',
        'USD_TRY','USD_ZAR']

# List of acquisition intervals
duration = ['M','W','D','H2','H1','M30','M10','M5']

# Repeat for each currency list
for k in kind:
 # Repeat for each acquisition sense list
 for d in duration:

  # get_data(Currency type, time range, start year, end year, start month, end month) (Get data up to 10 days)）
  file_path =  path + k + '_' + d +'.txt'
  print(file_path)

  # get_data
  get_histry_data(file_path,k,d,2005,2020,1,1)

  # Load and print saved data
  data = pd.read_csv(file_path)
  print(data

Acquisition of virtual currency data

The data acquisition for the virtual transit is available in a separate article, which you can read here if you are interested.

Conclusion

Now you can acquire millions of FX rates and build big data. We can use this data for back testing and machine learning.

If you are interested, please register with OANDA as well.

Let’s Encrypt

Certificates are available free of charge. That is by using Let’s Encrypt. Let’s Encrypt is run by ISRG, a US non-profit organization. Since it is a certificate that can be used free of charge, people who operate web pages on their own have no choice but to use it.

Install certbot

First, install certbot. It’s easy because you can do it with one command. As a prerequisite, use the yum command on CentOS.

sudo yum -y install certbot

Create a certificate with certbot

If you can type the certbot command, you can create a certificate with the following command. Although you can enter interactively, you can also create a certificate by entering only one line. Since certbot uses port 80, if there is a web server using port 80, it is necessary to stop it in advance.

sudo systemctl stop nginx.service
sudo certbot certonly --standalone -d example.work -m admin@example.work --agree-tos -n
sudo systemctl start nginx.service

By the way, if you are using port 80, this error will occur.

Problem binding to port 80: Could not bind to IPv4 or IPv6.

The certificate will have four files in “/etc/letsencrypt/live/example.work/”. Among them, we use the following two.
・fullchain.pem: certificate
・privkey.pem: certificate private key

Change nginx settings

Set the certificate created by the certbot command to nginx. Add SSL settings to the configuration file (/etc/nginx/conf.d/default.conf, etc.). The setting addition is a setting to redirect http to https and a setting to enable access with https with the certificate created this time .

Below are additional settings.

server {
    listen 80;
    server_name example.work;
    return 301 https://$host$request_uri;
}
server {
    listen       443 ssl;
    ssl_certificate         /etc/letsencrypt/live/example.work/fullchain.pem;
    ssl_certificate_key     /etc/letsencrypt/live/example.work/privkey.pem;
}

Certificate auto-renewal settings

If you can get the certificate on the command line, you can automatically update the certificate just by setting cron, and you can always support SSL. Depending on the environment, here we will set cron to execute commands as the root user. The SSL update can be done within 30 days of the expiration date, so I try to check it twice a month.

$ sudo crontab -u root -e
0 1 1 * * systemctl stop nginx.service
1 1 1 * * certbot renew
3 1 1 * * systemctl start nginx.service
0 1 20 * * systemctl stop nginx.service
1 1 20 * * certbot renew
3 1 20 * * systemctl start nginx.service

Conclusion

We’ve shown you how to turn your website into SSL for free. It’s easy, so if you have a web page, I think you should try it.

Ask AI for a picture that looks like a job

I couldn’t think of a picture that looks like a job, so I asked Chat GPT if there was a picture that looked like a job.
He gave me four suggestions!
　・Pictures like stopping a clock ・Pictures
　like baking bread　・
　Praying hands together

I got an answer that I never thought of. it’s interesting.

ask for specifics

What is a picture that stops the clock? I
asked him specifically about the first picture that would stop the clock, and instructed him to create an English sentence for Diffusion Stable to write. very. I got back a long sentence.

Here is the English translation.

“This painting depicts a young woman in a black dress standing stoically with her legs spread by a clock tower, staring into the horizon. Her face is a mask of determination. , holding the sides of the clock tower with both hands, one hand slowly reaching over the clock tower as her determined gaze is enveloped in darkness against the bleak landscape. The posture implies that she is metaphorically in a timeless space, not just in the moment, but in the larger, overarching story. We are trying to stop it in some way.”

It is a concrete content that reminds me of a woman desperately trying to stop time.
Sounds like she’s the kind of woman who doesn’t want the year to end. interesting idea.

Chat uses chat GPT on slack, so if you want to play, please check it out.

Let Diffusion Stable draw a picture.

I tried writing specific instructions for AI using Diffusion Stable.
“Painting that looks like a job” has become “Painting that stops the clock”, creating a picture that looks like the end of the world.

I wrote it using Diffusion Bee on M1 MAC.
If you want to draw a picture in the same way, check out this article!https://algo-ai.work/2022/12/17/post-1502/

Conclusion

It’s interesting that people can draw a picture with just a little assistance, from the word “work-like” to the AI ​​giving specific instructions to handle AI.
In the future, AI may be further hierarchized, and art may be born from hierarchical AI manipulation, such as AI instructing AI and AI operating AI according to that instruction. I think it will be a world where various things are completed.
The last few years have seen the development of AI.

Please follow me on Twitter if you don’t mind .

Google Colaboratory that can use GPU almost without environment construction is really amazing.

So, to get the most out of this amazing tool, we’ll show you how to run it automatically on a regular basis.

I use 3 Chrome extensions.

I also considered automatic execution with “Selenium + python”, but I could not log in to my Google account with Chrome opened as a robot browser, so I thought of another method.

Display execution time list on Google Colab

Any program can be executed periodically, but I will introduce a code that displays a list of execution times in Google Colaboratory so that you can see the time when it was executed periodically.

1. Go to Google Colab .
2. Select “New”.
3. Enter your python code.

import datetime as dt

# Obtaining the current time
now = dt.datetime.now()

# Output current time to file
with open("./result.txt","a") as f:
  f.write(str(now)+"n")

# Output a list of execution times
with open("./result.txt","r") as f:
  print(f.read())

PythonCopy

4. Press “” to execute.

If you execute it multiple times, the list of execution times will be displayed. We will run this program regularly.

Automate with UI. Vision RPA

Use UI.Vision RPA to automatically operate Chrome. The procedure is as follows.

1.  Install UI.Vision RPA .

2. Open UI.Vision RPA.

3. Click +Macro.

4. Click Record.

5. Click the “” button on Google Colab in chrome to record a macro.

When you run it, you can confirm that the operation is automatically recorded.

6. Click Stop Record.

7. Click “Play Macro” and confirm that it will run automatically.

8. Save the created Macro as html.

You can confirm that it is executed just by opening this saved HTML. This feature was not in Selenium, so it’s an excellent feature.

Regular execution with cronTab

You should be able to run Google Colab just by opening the HTML file saved up to this point. After that, if the HTML file can be opened at the specified time, regular execution will be possible. The extension used there is cronTab . By using this function, you can open a specific path (URL, etc.) periodically. The procedure is as follows.

1.  Install cronTab .

2. Open cronTab.

3. Make settings to open the HTML file from earlier. Enter the location (path) of the file in the URL. By adding “?direct=1” at the end here, you can eliminate the need to press “OK” after opening the HTML file and automate it. Conversely, if this postscript is not present, it is necessary to press the “OK” button, and automatic execution cannot be performed. In addition, the example below is an example of automatic execution every 2 minutes.

Automatically close tabs with Tab Wrangler

cronTab will automatically open a new tab. Therefore, tabs will be generated more and more unless they are closed automatically. So, let me show you how to automatically close tabs in Tab Wrangler.

1.  Install Tab Wrangler .

2. Open Tab Wrangler and configure it. I think the settings are a matter of personal preference.

That’s it, you can run Google Colab regularly with just the Chrome extension. Now you can study Deep Learning regularly, and your research will progress. Of course, it is also possible to forecast time-series data on a regular basis. AlgoAI regularly predicted and tweeted time series data. 

Follow me on Twitter if you ‘d like.

How to install Diffusion Bee

I will show you how to install Diffusion Bee.
Go to the Diffusin Bee website, download the dmg file and install it.
It’s the same as using a normal application, so it’s great because you can easily try it without creating a program.
Other than this software, I couldn’t find any software or programs specialized for M1 at present.

Access the link below.

Click Download for macOS.

Click MacOS Apple Silicon to download the dmg file and install the downloaded file.

How to use Diffusion Bee

When you open the application, a screen for entering text will appear. Just enter the drawing spell you want to write here and click Generate to draw with the default model. I was blown away by how easy it was to try Diffusion Stable.
Even with Google Colabo, it took several minutes to draw with GPU, but with DiffusionBee of M1 MAC, I could draw a picture in tens of seconds.
Below, I drew a white dog on green.

The result is a picture like the one below. It’s an original painting, a non-existent dog and place.

Diffusion Beeに新たなモデルを追加する方法

How to add a new model to Diffusion Bee

You can create your own models in Dream Booth or use non-default models. Here is a list of models that
are likely to be samples, so please refer to them. You can source the ckpt file and use any model you like.

In Diffusion Bee, click the three lines in the upper right and select Settings.

Under Custom Model, click Add New Model.
Click to select the previously downloaded ckpt file.

Once you have set the Custom Model, you can select the Custom Model from Options in Text To Image, etc. You can easily try various models, so your ideas will expand. With Image To Image, I feel that it is possible to develop services based on fixed words, and there is great potential for service development.

Finally

I introduced how to easily try Stable Diffusion on M1 MAC.
You can easily try it without writing a program, and the processing is fast and wonderful.
If you have a MAC, give it a try.
If you want to create an original model, you can use Dream Booth to create an original AI with about 10 photos.