All posts by Kim Nyholm

Install maven and Java SE Development Kit

I use the script install-java8jdk.sh to install maven and Java Development Kit. To enable installation of JDK without prompt for license agreement the below command is issued

sudo debconf-set-selections <<<'debconf shared/accepted-oracle-license-v1-1 select true'

The installation can be checked with the command ‘mvn -v‘ which will result in an output like:

vagrant@ubuntu:~$ mvn -v
Apache Maven 3.0.5
Maven home: /usr/share/maven
Java version: 1.8.0_91, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-oracle/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.13.0-87-generic", arch: "amd64", family: "unix"

Get IMAP mail embedded images in php

There is plenty of php snippets , showing how to get mail from an imap server. Even though many illustrates important concepts most are either faulty or incomplete. The file imapClient.php implements a complete simple php imap web client in less than 300 lines with the following features:

  • connecting to the server using imap_open()
  • finding messages to download with imap_search()
  • downloading messages including embedded images and attachments using imap_headerinfo(), imap_fetchmime(), imap_fetchstructure() and imap_fetchbody()
  • decoding messages and filenames into utf8 using charset information in message
  • formatting messages to html with embedded images and attachment links
  • html image references in the mail text src=”cid:…” are substituted with image urls.
  • closing server connection using imap_close()

The code is a good starting point if you want to pull in mails in php for further processing.  Errors are handled but not reported. Some points you would address are better styling and cleanup of attachments on the web-server after a session. On the mail server mails should be deleted or marked as read. And certainly you wouldn’t normally allow a client to download any attachment to your web-server for security reasons.

The code itself is pretty much self explained, so I won’t add any details for now. The tricky parts are to

  • do the recursion on the structure from imap_fetchstructure()
  • interpret the mime information
  • decode the data with the right decoder
  • handle more attachments with same name

The mail client is self contained, i.e. only the file imapClient.php is required. Point your browser to imapClient.php, to check it out. For attachment download to work correctly, your web-server needs permissions to create the attachment directory in the same directory as the imapClient.php file.

A good implementation of an imap mail reader is the Drupal mailhandler module. If you download the module, relevant source is in the file MailhandlerPhpImapRetrieve.class.php.

Da de humanistiske værdier røg ud med badevandet i Danmark

Jeg havde besluttet mig for at stemme nej til at ophæve retsforbeholdet til EU-stemningen. Hvem tror på EUs politikere og en EU-statsmagt? Danmark har en lang tradition for højskoledemokrati og er den bedste retsstat i verden.

Så gik det danske politi i aktion:

  • og stopper fredelige demonstranter. Fordi den kinesiske præsident kunne blive fornærmet hvis han så at der er danskere som er uenige med Kinas politik i Tibet.

Kære politi, hvad sker der for jer, I burde da om nogen kende loven.

Dernæst melder politikerne klart ud:

  • det er ok at bure flygtningene inde tidsubegrænset
  • og kun rimeligt at konfiskere deres værdier så de kan betale for deres ophold

Internering og konfiskation bringer dårlige minder frem.

Hvad sker der for dig Danmark?

Jeg er nødt til at erkende at Danmark har mistet troen på de humanistiske værdier som jeg hylder. Jeg giver min stemme til EU.

 

Replace your Drupal database

It may be useful and fast to replace a Drupal database directly from bash/mysql, with the following commands, where <database> is the name of the database:

  • delete database
    mysql -u ada -pLovelace
    drop database <database>;
    exit
  • create database
    mysql -u ada -pLovelace
    create database <database> character set utf8 collate utf8_general_ci;
    exit
  • import database from ‘Backup and Migrate’ database file
    mysql -u ada -pLovelace -D <database> <database.sql

Useful GIT commands

When working on both Windows and Linux platforms be sure to add proper line ending support, i.e. work with only LF in repositories.

  • On Windows work with CRLF, git config --global core.autocrlf  true
  • On Linux work with LF, git config --global core.autocrlf input

If you want to force a repository to always use specific line endings, add the setting to a .gitattributes file.

The table below shows some useful git commands. In the command column, a quoted word is parameter, e.g. ‘repo’ could be origin. The local and the remote repository are called repo and remote.

CommandDescription
git clone 'directory1' 'directory2'Clone remote at 'directory1' to repo at 'directory2'.
git remote -vList urls/directorires of all remotes.
git add 'remote' 'directory'Add the remote at 'directory' to the repo with nickname 'remote'.
git fetch 'remote' 'branch'Update cache of repo with remote branch.
git pull 'remote' 'branch'Merge content of remote branch into repo (shortcut for git fetch.. & git merge... ).
git merge 'branch'Merge content of local 'branch' into current branch.
git merge 'remote'/'branch'Merge content of local cached remote/branch into repo.
git push 'repo' 'branch'Merge local repo into remote. Don't try to push into non bare remote, instead make a pull from the remote.
git commit -m “commit comment”Commit changes with the comment.
git add .Prepare all modified/new files for commit.
git add 'file'Prepare 'file' for commit.
git branch 'branch'Create branch 'branch'.
git checkout .Undo changes on all files.
git checkout 'file'Undo changes on 'file'.
git checkout 'commit'Checkout specific commit. To go back to HEAD use git checkout 'branch'
git checkout 'branch'Change working directory to head of branch
git reset HEAD~1Remove latest commit, leaving changes in working directory.
git reset --hard HEAD~1Remove latest commit and undo changes in working directory.
git stashSave changes in working directory.
git stash listShow list of saved stashes.
git stash popApply changes from last stash.
git config --listList git configuration.
git clean -dxfClean up non controlled files in working directory (use -dxn to do a dryrun)
git log --statShow commit log, inclusive which files have been changed
git commit --amendAdd changes to last commit. Don't use if you have already pushed the last commit to a remote.

Configuring ftp server

An ftp server is being used by Drupal when installing new modules. For some reason Drupal uses ftp to copy uploaded files to the filesystem, even when the filesystem is on the same system as the web server.

Furthermore some development tools running outside the vm, e.g. NetBeans on the host is using ftp to synchronize files between the vm and the host.

I use vsftpd as ftp-server. The script install-vsftpd.sh installs vsftpd and generates vsftpd.conf.

Configuration of Virtual box and VSFTPD

It is configured in /etc/vsftpd.conf. Compared to default setup, the following parameters are changed:

  • local_umask  to 022. This ensures that the www-data user is able to read files uploaded by other users.
  • write_enable to yes. This enables ftp users (e.g. NetBeans) to upload files.
  • listen to yes. This ensures we are running ipv4. Using ipv6 may cause issues on AWS EC2.

Port forwarding for the virtual box is not required since we have given the vm its own ip and the domain name ubuntu.dev.

You can learn more on vsftpd setup in how-to-setup-and-use-ftp-server-in-ubuntu-linux, when reading about “Stand-alone vs Normal FPT mode”, have in mind that this setup uses Stand-alone mode.

Configuration of Amazon EC2 and VSFTPD

To configure vsftp in the EC2 instance to be accessible from the outside world you additionally to

  • get a (sub)domain for the EC2 instance. In case you do not have a domain, you may replace the options mentioned below for vsftpd.con with
    • psav_addr=YOUR.PUBLIC.IP.ADDR
      pasv_addr_resolve=NO
  • create a security group for the EC2 instance with the following inbound rules
    • allow tcp port 20-21
    • allow tcp ports 1024-1048
  • add the following settings to /etc/vsftpd.conf
    • pasv_enable=YES
      pasv_min_port=1024
      pasv_max_port=1048
      pasv_address=YOUR.DOMAIN.NAME
      pasv_addr_resolve=YES
      port_enable=YES

 

Configuring and using VIM

Vim is an advanced text editor for use on linux. If you have not used it before, you will need one cheat sheet or some vim tips and tricks to find your way around.

When I use Vim I add a few features:

  • show dark text on light background
  • show line numbers
  • show trailing whitespaces
  • use spaces for tabs
  • set tabstop to 2 spaces
  • set shift width to 2 spaces
  • read files as unix files (show windows/dos CRLF endings as ^M)

This is done by creating the file  ~/.vimrc with the following content:

set t_Co=256
highlight Normal ctermfg=black ctermbg=white
set number
match ErrorMsg ‘\s\+$\|\t’
set expandtab
set tabstop=2
set shiftwidth=2
set fileformats=unix

To create the file I use the script install-vim.sh.

If you have a file loaded into a VIM buffer with windows/dos CRLF endings, it can be converted to unix format by executing the commands, as explained on the Vim Tips Wiki:

:e ++ff=dos
:setlocal ff=unix
:w

Bash command line editing

Using the bash command line may seem difficult at first, if you are coming from a Windows world. However a few editing commands key strokes your life easier. A detailed Command Line Editing sheet may help, but you really only needs a few commands . Below C- means the Control key and M- the Meta (or esc) key:

Key Action
C-a (C-e) Move to the start (end) of the line
M-f (M-b) Move forward (backward) a word
C-b (C-f) Move back (forward) one character
C-k (u) Kill the text from the current cursor position to the end (start) of the line
M-d (M-DEL) Kill from the cursor to the end (start) of the current word, or if between words, to the end (start) of the next word
C-d  (DEL) Delete the character underneath (left of) the cursor
C-_ Undo the last thing that you did
C-l Clear the screen, reprinting the current line at the top
C-y Yank the most recently killed text back into the buffer at the cursor

How to send mail from apache2

To send mail from apache2 using the PHP mail() function requires a Mail Transfer Agent to be installed. Several exists, I use postfix.

The following assumes you can access your web server at ubuntu.dev. This will be the case if you a setup as described for the host. Replace with your hostname if you have an other installation.

How to install and configure the Mail Transfer Agent is described detailed in the Ubuntu postfix guide. Unless you have some specific needs, don’t bother to read it, most is about setting up parameters, but defaults works fine.

The short version is to install the packages postfix and mail-stack-delivery with the commands:

sudo apt-get install postfix
sudo apt-get install mail-stack-delivery

During installation you will be prompted for a few parameters, where the values should be quite obvious from the questions.

This is scripted in install-postfix.sh, where the required parameters are also supplied.

To send an email from your web-site you make a call in your php code like:

mail('receiver@domain', 'mail subject', 'mail body');

As starting template or to test that mail can be send, use testmail.php, which you activate from your browser with

http://ubuntu.dev/testmail.php?to=receiver@domain&subject=header&message=body

Replace receiver@domain, header and body with your values.