johna's blog

Another pointless project - the programmable digital watch

2025-01-20T12:00:00+10:00

I've come up with yet another pointless project.

Recently, I've rekindled my interest in digital watches after having found my old calculator watch from the 1990s.

With mobile phones, there's no longer much need for a watch, let alone a calculator watch, and I was trying to think of what features might make a watch more appealing in the current age.

To be honest, I couldn't think of any new features or good reasons to wear a watch other than for fashion reasons, or for people who want to be able to check the time quicker and more convenient than reaching for their mobile phone.

But you can still buy a new calculator watch in 2025, so that indicates to me that there is a market for pointless watches for hobbyists. So what would be of interest to hobbyists?

The only thing that I could come up with was being able to program your own watch. I know you can get smart watches and have apps for any purpose, or create your own app, but I was thinking of a much simpler, more convention digital watch that uses a typical watch battery that lasts for years rather than needing to be charged every day.

It would have a simple BASIC-style programming language that would have minimal features and commands but enough to do anything a watch needs to do and more.

How would you program it? You could develop in a desktop application or browser-based editor with a built-in emulator, and then compile into some sort of minimalist opcode and transfer to the watch by a simple serial two-pin cable. No USB as that might add to the complexity of the computing power needed for the watch.

As a proof-of-concept I started working on a web-based emulator with a JavaScript library of functions, similar to what I would expect the watch would have. For my POC, developing your own programs is in the JavaScript language simply because I didn't want to go to the lengths of creating my own compiler.

I have no idea the available computing power of a digital watch so I don't know if this is actually possible. For the POC, I kept things pretty minimal. I chose a display consisting of 4 rows capable of alphanumeric characters plus a row of common icons (AM, PM, 24HR, DST, ALM, SNZ, CHIME). The font is made up of 5 x 5 pixels. However, a fully customisable pixel display would be more flexible if possible.

The top and bottom rows are 13 characters long and are intended to indicate what each of the four buttons do, as their functions would not be printed on the watch, and will vary depending on the mode and settings of the watch.

I liked the idea of the different modes of the clock being stored in self-contained modules so did this for my example.

This is very much a work-in-progress. The JavaScript library is only for the purpose of emulating what the watch would do.

It's been quite interesting thinking of how to make the various watch functions work and the potential problems associated with certain situations, such as if an alarm was to go off while in the middle of setting the time.

If you are reading this post, my question to you is if you had access to a programmable digital watch, what features would you like to create?

UPDATE: turns out there is already something similar to this out there. Sensor Watch is a new circuit board that fits inside a Casio F-91W watch housing and has an ARM processor. There is various firmware to download and a framework for creating your own. Updates are through USB and despite being a powerful processor, the battery can last 18 months. As good as this is, and much more powerful than I imagined possible, the programming part is more complicated than I would have aimed for, and the F-91W display is very limiting.

UPDATE 2: Back in 1984 there was the Seiko UC-2000 -- a digital watch with a docking station with keyboard and even a printer. These were programmable in Microsoft BASIC. Read about it on Hackaday or find some interesting information here including an emulator.

Try it

Here is my incomplete first attempt at the proof-of-concept.

The main clock mostly works, setting the date and time works but has issues due to problems with the JavaScript library, and the stopwatch works pretty well. I will keep working on these and the other functions like alarm, dual time and countdown timer.

Inspect the page code to see how it works.

Once the code is in a better state I might make a package that can be downloaded and will be easier to do some development for.

24HR AM PM DST ALM SNZ CHIME

Proposed/Example Command reference

LET / CONST / VAR
Use these JavaScript commands to declare variables

ADDSHOWDATE(name, datetimevalue, row, column, format)
Shows a date on the specified row, starting at the specified column, in the specified format.

ADDSHOWTIME(name, datetimevalue, row, column, format, clockmode)
Shows a time on the specified row, starting at the specified column, in the specified format, with am/pm/24 hour indicator, and update when the time or date changes.
Formatting string allows you to control what parts of dates and times are shown, how many digits, and whether they should flash or not (useful for a set mode).

REMOVESHOWTIME(name)
Stop showing and updating a time previously set with SHOWTIME.

PRINT(row, col, stringvalue)
Prints alphanumeric characters on the specified row starting at the specified column.

ONBUTTON(buttonnum, handler)
When a button is pressed (numbered 0 to 3 clockwise) call the specified handler function.

ONTICK(name, handler)
Add an event handler function that will be called every second. Set handler to NULL to stop event handling.

ADDCLOCK(name, datetimevalue, increment)
Add a datetime object that the watch will automatically increment based on the specific increment. The normal clock and stopwatch would typically increment by 1 second, a countdown timer would increment by -1 seconds.

REMOVECLOCK(name)
Remove a clock from being updated by the watch.

CLEAR(row = NULL)
Clear the screen or a specific row.

DISPLAY(icon, status)
Show or hide one of the built-in icons (am, pm, 24hr, dst, alarm, snooze, chime).

DATE FUNCTIONS

I added a couple of my own functions, but for this POC I am mostly using JavaScript date object functions.

SETDATE()
SETTIME()

GETHOURS()
GETMINUTES()
GETSECONDS()
GETMILLISECONDS()
GETYEAR()
GETMONTH()
GETDATE()
GETDAY()

SETHOURS()
SETMINUTES()
SETSECONDS()
SETMILLISECONDS()
SETYEAR()
SETMONTH()
SETDATE()
SETDAY()

TODO

I need to add a function to sound a chime or alarm, and possibly handle stopping an alarm rather than make the programmer handle this.

It would be nice to be able to develop games for the watch so that would need the ability to display graphics, play sounds, and have an event handler quicker than every second.

Converting dBase IV programs to run in the browser and Text User Interfaces (TUI)

2024-09-13T12:00:00+10:00

Recently, my mind drifted back 35 years ago. A time when I was developing business applications in dBase for DOS and Windows was only starting to become a thing.

At the time my colleagues and I joked about business applications being developed for Windows. Imagine having to use a mouse to move around the screen and click on things instead of just using the keyboard to do everything?

Time has, of course, proven that Windows is perfectly suitable for these types of applications but my reminiscence did prompt me to look at some of my old applications and think about whether there is still a place for text-based and keyboard-oriented applications. Are they better or faster to use for certain types of business applications?

Then in a moment of insanity, I thought maybe the world might need or want a JavaScript framework for text-only, keyboard-oriented web applications.

That lead me to wonder how would my old applications would run in the browser. One way to find out: try and convert one to run in JavaScript.

So I created a HTML page with a fixed-width (monospace) font, created an element to house my 80x25 screen, and then copied across my old dBase program file to work on.

I had to strip out all the database code as this was purely a test of the user interface and any database functionality would, of course, have to be done separately on a server.

Converting dBase code to JavaScript is simple: I just had to change things the various dBase commands to the equivalent JavaScript. Like the WHILE, DO CASE and IF commands. The comparison operators = and # need some attention. String functions need to be changed. A lot of semi-colons and curly braces need to be added.

But to make the dBase code work the same I needed to write some new functions. SAY prints some text at a specific row and column so I needed a way to do this. I opted for an array of strings for each row to act as the screen memory and this would be modified by the new say function and then the screen redrawn each time it changed.

GET adds a control where data can be entered and needed some thought. I could use a HTML INPUT control or I could listen for keyboard input and write my own input control. I opted for the HTML INPUT control initially, mainly because it would be easier and quicker. So the GETs are now stored in an array and HTML controls inserted into the "screen" when the screen is redrawn.

Now dBase IV is a procedural system where code execution waits for things like data entry and keypresses before it continues, but JavaScript is more event-driven and doesn't like to wait in loops for something to happen. To make it work I needed to add async functions and promises.

I wrote a function equivalent of dBase's READ to handle form entry and added READKEY and LASTKEY equivalent functions to store the key(s) that were used to exit the form. I also converted the JavaScript key codes to the original dBase values.

Various other functions added and changes made, and here is the result:

Open in new window

It's a working application! And it actually works reasonably well except the data entry and key handling is not quite the same as dBase and has some issues. If I were to make a framework for these kind of applications I think it would be better to not use HTML controls and write my own. Or possibly embrace HTML form elements fully.

Ideally any framework would be event-driven to better work with JavaScript.

So are business applications better in a text and keyboard-orientated environment than in a GUI like Windows? If the application requires typing and you know what key does what then it can be quicker to enter data and complete a form. Typing into a normal web form with text inputs and selects can be a little slower as you alternate between keyboard and mouse, and most web forms have a button to submit the form, not a specific key, so you need to switch to the mouse or use the tab key to get to the submit button and then an extra keypress to operate the button using the keyboard.

So I can kind of see a use for a framework that is focused on quick data entry using the keyboard. I have no idea if something like this already exists but will look in to it further.

UPDATE: I came across TuiCss, a CSS and JavaScript library that can be used to make MS-DOS like applications, based on the Turbo Vision framework.

How to set up a debugging using the Turnkey Linux LAMP stack and VS Code

2023-12-19T12:00:00+10:00

In my previous post I gave instructions on hosting a website and database on the Turnkey Linux LAMP stack. This post is a continuation and covers setting up debugging the website in Visual Studio Code.

First step is to modify the php.ini file on the server. You can do this in Webmin. In the System menu press PHP Configuration. On the row for "Configuration for mod_php" press Edit Manually to open the file for editing.

Then add a configuration section for Xdebug, as follows:

[xdebug]
xdebug.mode=develop,debug
xdebug.discover_client_host=1
xdebug.client_port = 9003
xdebug.start_with_request=yes

After saving this change you will need to restart the Apache Webserver. This can be done in Webmin. Press Apache Webserver in the Servers menu and then press the apply changes icon button near the top right corner.

Then start VS Code and add the extensions PHP Extension Pack by Xdebug which includes PHP Debug and PHP IntelliSense and SFTP by Natizyskunk.

You need PHP on your workstation for these extensions to work, so download it from windows.php.net. The LAMP stack currently includes PHP 7.4 so I downloaded PHP 7.4.32 NTS Windows 64-bit. Once downloaded and uncompressed move it to your preferred location, you then need to tell the extensions the path to php.exe with the following three settings in settings.json (use your own path, not mine and ensure php.exe is included):

"php.executablePath": "D:\\Documents\\php-7.4.32-nts-Win32-vc15-x64\\php.exe",
"php.debug.executablePath": "D:\\Documents\\php-7.4.32-nts-Win32-vc15-x64\\php.exe",
"php.validate.executablePath": "D:\\Documents\\php-7.4.32-nts-Win32-vc15-x64\\php.exe"

You can now configure SFTP. Press Ctrl+Shift+P to bring up the command prompt and type SFTP: config. You can then enter a configuration similar to the following, using your website details, and FTP user and password.

{
    "name": "My Website",
    "host": "mywebsite.local",
    "protocol": "sftp",
    "port": 22,
    "username": "mywebsite",
    "password": "abc123",
    "remotePath": "/var/mywebsite",
    "uploadOnSave": false,
    "useTempFile": false,
    "openSsh": false
}

Next configure the debug settings.

{
    "name": "Listen for Xdebug",
    "type": "php",
    "request": "launch",
    "port": 9003,
    "pathMappings": {
        "/var/mywebsite": "${workspaceFolder}/"
    }
}

You are now ready to debug. Press F5 to start listening for Xdebug connections and set breakpoints where needed.

If you need to set up URL rewriting, see my next post How to enable URL rewriting using the Turnkey Linux LAMP stack.

How to set up a website and database using the Turnkey Linux LAMP stack

2023-11-18T12:00:00+10:00

If you need to host your own website for the purposes of web development, Turnkey Linux LAMP Stack is an easy to install all-in-one solution that you can set up on a spare computer or a VM (Virtual Machine).

I recently wanted to do this so I could locally test a website project I have been working on, and do PHP debugging. I had a spare old Celeron PC that I used for this purpose.

Step one is to download the ISO and install it on a computer or VM. I used Rufus to copy the ISO file to a USB drive but you don't need to do that if installing on a VM.

Installation of Turnkey Linux is simple and the questions that get asked during guided to install should be easy to answer. Once installed you can log in with root and the password you chose during installation and then set up your network with either DHCP or a static IP. I chose a static IP so that I could set up a local domain name I could use when accessing my website locally.

In Windows you can set up a local domain by editing the file C:WindowsSystem32driversetchosts. Note that you will need to run Notepad as an administrator to be able to save this file. Then just add a new line with the static IP address you chose and the domain name, for example:

192.168.1.100 mywebsite.local

For many of the next steps you will need to log in to Webmin, a built-in website that allows you to administer the server. The default address is https://192.168.1.100:12321 (change the IP address to the correct address for your server). You will need to bypass a security warning as the website uses an untrusted self-signed certificate. The user name is 'root' and the password is the one you chose during installation.

Next step is to create a folder for your website. Go to the Tools menu and then press File Manager.

The default websites are located in the var folder, and I would suggest your website(s) also be created there, so select Create new directory in the File menu near the top right corner of the page, and type in a name for your website, eg "mywebsite".

You can now go to the Servers menu and press Apache Webserver, then press Create virtual host. There are many options for setting up a website, but I entered 80 for Port, chose my newly created folder for Document Root, and entered my local domain for Server Name.

You can now navigate to your local domain in a web browser and you should get a "Forbidden" error message as you don't yet have any files or pages to serve.

Most websites have a database, and the LAMP stack includes MariaDB, a MySQL compatible database server.

We can now create a database and a database user.

From the Servers menu, press MariaDB Database Server, and then press Create new database. On the form enter a name for your database and optionally choose any other settings you may need, or just leave the defaults.

Once the database is created, press User Permissions and then Create new user. Enter a username and password and select Permissions that your user will need (I selected all of them).

After the user is created you can give the user access to your database. Press Database Permissions and then Create new database permissions and then select your new database and enter your new user name and then create the permission.

You can check you have database access by going to https://192.168.1.100:12322 (change to your IP address) and logging in with your new database user. Once logged-in you can also import or create your database objects and data if you are ready.
Although you can manage files using the built-in file manager, you will most likely want FTP access so you can deploy code changes from your web development IDE. For this you will need to go to the System menu and create a new user. Enter a username, select your website folder as the Home directory, and enter a new password as the Normal password.

Once your user is created you can connect using SFTP to your server's IP address or local domain and using the user name and password you just created. You will need to change ownership to the folder to your FTP user and also change permissions.

In a future post, I will document how to configure Xdebug and debug from Visual Studio and/or VS Code, and deployment via SFTP.

Intermittent "Unable to read data from the transport connection: net_io_connectionclosed" errors

2020-05-06T12:00:00+10:00

I have been facing a problem with an ASP.NET application that sends email using a remote mail server.

Some emails failed to send, and the error message logged was "Unable to read data from the transport connection: net_io_connectionclosed".

The problems were very intermittent and started occurring after a change of mail server. It was not easy to reproduce the problem.

I didn't get much help from the people supporting the new mail server, although they did confirm that the error was also logged at their end.

I also spent plenty of time searching for possible causes and trying various solutions none of which worked.

Sending was done using System.Net.Mail library and there was some suggestion in my research that this has some issues and is actually deprecated (there is some confusion over that though). Microsoft is recommending using an open-source library called MailKit instead.

So I decided to do some experimentation and rather than add any new dependencies I temporarily changed some of my code to use the outdated but believed to be more reliable System.Web.Mail library. The problem went away – no more errors!

Unfortunately System.Web.Mail couldn't easily do one thing I need to do (embedded images that don't also appear as attachments), so I opted to rewrite most of my mail sending code to use MailKit. MailKit can do all that System.Net.Mail and System.Web.Mail can do and more, and it's not hard to switch over to it.

There's been no further email errors with MailKit so I would suggest that you move away from System.Net.Mail if you ever face this error message.

500 Internal Server Error after migrating from IIS 7.5 to IIS 10

2019-11-04T12:00:00+10:00

As support ends for Microsoft Windows Server 2008 in January 2020, I have recently gone through migrating some websites to a new server running Windows Server 2016 and IIS 10.

The websites were migrated with the Web Deploy Tool.

Some of the websites did not work, and simply brought up a 500 Internal Server Error, bypassing custom error pages. This is always a sign that the cause of the error might be in the web.config file.

After a little experimenting with commenting out certain sections of the web.config file, I found two sections that had compatibility issues between IIS 7.5 and IIS 10.

The first was the "staticContent" section. I had used this to allow IIS to server .woff and .svg files. This is no longer needed for those types of files in IIS 10 so the section could be removed.

The other problem was where the IIS URL rewrite module had been used to automatically redirect from HTTP to HTTPs.

This module is not installed in IIS 10 by default.

You could download the URL Rewrite extension (https://www.iis.net/downloads/microsoft/url-rewrite) but if you are only using this to redirect from HTTP to HTTPS then IIS 10 has an easier way of doing this:

• Open the IIS console and select the website
• On the "Actions" pane you will see the "HSTS" option below the "Configure" section
• Enable HSTS and "Redirect Http to Https"

(Source: https://www.hametbenoit.info/2018/10/22/windows-server-2019-you-can-now-easily-redirect-http-request-to-https/)

There may be other causes of these problems too but they should be reasonably easy to identify by commenting out sections of the web.config file one at a time and testing, and a little research with your favourite search engine.

A jQuery plugin for a fixed table of contents

2018-10-12T12:00:00+10:00

Some long web pages have a handy feature where a table of contents are shown beside the content which remains fixed.

A good example of this was the Bootstrap 3.x documentation pages. Once you scroll down past a certain point the table of contents position changes to fixed so it is always visible.

There is a problem with this particular implementation though. If the table of contents list is long and exceeds the height of the browser window, then it is impossible to see the bottom part of the table of contents.

In Bootstrap 4.x (example) they changed the way tables of contents were displayed to avoid this problem. They are now always fixed position and if the content cannot fit in the vertical space they show a secondary scroll bar just for the table of contents.

This works well enough, but I was looking for a more automatic method closer to the way it was done on the Bootstrap 3.x pages for my Bootstrap Customizer website, but with the ability to be able to scroll to see any content that wouldn't fit in the browser window.

This is quite a challenging task as the requirements are a little complex:

1) The TOC (table of contents) displays and scrolls as normal until you scroll down to the end of the TOC. At this point the TOC displays as fixed with the bottom of TOC at the bottom of the browser window.

2) When scrolled down to the end of the page, the TOC should not be displayed over the footer, and the end of the TOC should display at the end of it's container.

3) When you scroll up the TOC should also scroll up. This should only happen up to the point where the top most item in the TOC is at the top of the screen, or has reached it's original natural position.

Based on these requirements, I've been working on a prototype.

(function ($) {
	$.fn.scrollfixed = function (options) {
		var settings = $.extend({
			breakpoint: 0
		}, options);

		var obj = this;
		var isFixed = false;
		var atBottom = false;
		var lastScrollTop = -1;
		var contents_top = obj.offset().top;

		var contentTop = 0;
		var contentTopOriginal = obj.offset().top;

		function handleScroll() {
			var contentHeight = obj.outerHeight(true);
			var containerTop = $('#' + options.container).offset().top;
			var containerHeight = $('#' + options.container).height();
			var containerBottom = containerTop + containerHeight;
			var viewportHeight = $(window).height();
			var scrollTop = $(window).scrollTop();

			var scrollingUp = scrollTop < lastScrollTop;
			var scrollingDown = scrollTop > lastScrollTop;

			var nextIsFixed;
			var nextContentTop;

			if (window.matchMedia('(min-width: ' + options.breakpoint + 'px)').matches && containerHeight !== contentHeight) {

				if (scrollTop > contentTopOriginal) {
					nextIsFixed = true;

					if (contentHeight <= containerHeight && (containerBottom - scrollTop - contentHeight) < 0) {
						if ((containerBottom - scrollTop - contentHeight) < 0) {
							nextContentTop = (containerBottom - scrollTop - contentHeight);
						}
						else {
							nextContentTop = 0;
						}
					}
					else {
						nextContentTop = contentTop;

						if (scrollingDown) {
							if (scrollTop > containerBottom - (contentTop + contentHeight)) {
								nextContentTop += (lastScrollTop - scrollTop);
							}
							else if ((contentTop + contentHeight) > viewportHeight) {
								nextContentTop += (lastScrollTop - scrollTop);
							}

							if (nextContentTop < (viewportHeight - contentHeight)) {
								nextContentTop = (viewportHeight - contentHeight);
							}
						}
						else if (scrollingUp) {
							if (contentTop < 0) {
								nextContentTop -= (scrollTop - lastScrollTop);
							}

							if (nextContentTop > 0) {
								nextContentTop = 0;
							}
						}
					}
				}
				else {
					nextIsFixed = false;
					nextContentTop = 0;
					task = 'fits???';
				}
			}
			else {
				nextIsFixed = false;
				nextContentTop = 0;
			}

			if (nextIsFixed !== isFixed) {
				obj.css("position", (nextIsFixed === true ? "fixed" : ""));
				obj.css("left", (nextIsFixed === true ? obj.offset().left + "px" : ""));
			}

			if (nextContentTop !== contentTop || nextIsFixed !== isFixed) {
				obj.css("top", (nextIsFixed === true ? nextContentTop + "px" : ""));
			}

			isFixed = nextIsFixed;
			contentTop = nextContentTop;

			lastScrollTop = scrollTop;
		}

		$(window).scroll(function () {

			clearTimeout(handleScroll._tId);
			handleScroll._tId = setTimeout(function () {
				handleScroll();
			}, 10);
		});

		handleScroll();

		return this;
	};
}(jQuery));

This is done as a jQuery plugin, and can be called with parameters "breakpoint" (the pixel min-width at which the positional changes should take effect, so you can disable this functionality on small screens if needed) and "container" (the ID of the container object that should be used to calculate minimum and maximum positions - this would usually be something like a Bootstrap column).

$("#myTOC").scrollfixed({ breakpoint: 768, container: 'myTOCContainer' });

Demo

This achieves all of the objectives but there's still a few minor issues with this code which I need to improve.

Change History

12-Dec-2020 - Code revised and improved, demo added

How to add AJAX suggestions to yairEO's Tagify tag input component

2018-05-25T12:00:00+10:00

This post applies to Tagify v1.3.1 which has since been replaced by v2.x.

I've since written my own very simple tag entry plugin.

When looking for a tag input component, like the one used on Stack Overflow, I came across Tagify.

This is a great implementation of a tag component, being very lightweight and using in-built browser behaviour where possible, for example rather than creating an auto suggest function like most other similar components, the creator uses a HTML 5 datalist element.

I chose to use this component for one of the websites I am developing, but I need to make some changes to make it work how I want it to.

The first change is to get suggestions via AJAX rather than a fixed whitelist. A fixed whitelist is fine for most applications, but in my case there could be thousands of possible tags and I was concerned about putting the entire list on the client.

Some other changes that I need to make will be covered in a future post. For example, as my website uses Bootstrap I want the tag input component to look like a native Bootstrap control.

Note that if you use this component with Bootstrap that you will have to hide the original INPUT or TEXTAREA control yourself as Tagify attempts to hide it using CSS (tags input, tags textarea { display: none; }) which is overriden by Bootstrap's CSS.

For graceful degradation, hiding this with CSS is not ideal, so using JavaScript is better anyway. I simply use the following after initialising Tagify on the control:

$("#myinput").tagify(); //initialise Tagify
$("#myinput").hide(); // Hide the original control

My modified version of Tagify is at the end of this article. I've added comments to indicate additions and modifications to the original code.

There's an additional setting, suggestionsUrl for the URL of the service that will provide suggestions via HTTP GET. The search term will be appended to this, so you may need to include part of your querystring in the URL, eg. "https://domain.com/services/tagify.php?tag=".

Tagify's autocomplete setting also needs to be set to true, and you should not include values for both whitelist and suggestionsUrl.

Note that the HTML 5 datalist auto suggest works by finding the search term anywhere inside the list of suggestions, so you should do the same in your service that provides suggestions. ie. that means you shouldn't provide suggestions based on the first characters only.

Your service should return a JSON result similar to this example (based on input of "ab"):

["ABAP","ABC","ABC ALGOL","ABSET","ABSYS"]

Okay, here's the code...

/**
 * Tagify (v 1.3.1)- tags input component
 * By Yair Even-Or (2016)
 * Don't sell this code. (c)
 * https://github.com/yairEO/tagify

 * Modifications for AJAX suggestions by John Avis 2018-05-25
 */
;(function($){
    // just a jQuery wrapper for the vanilla version of this component
    $.fn.tagify = function(settings){
        return this.each(function() {
            var $input = $(this),
                tagify;

            if( $input.data("tagify") ) // don't continue if already "tagified"
                return this;

            tagify = new Tagify($input[0], settings);
            tagify.isJQueryPlugin = true;
            $input.data("tagify", tagify);
        });
    }

function Tagify( input, settings ){
    // protection
    if( !input ){
        console.warn('Tagify: ', 'invalid input element ', input)
        return this;
    }

    this.settings = this.extend({}, settings, this.DEFAULTS);
    this.settings.readonly = input.hasAttribute('readonly'); // if "readonly" do not include an "input" element inside the Tags component

    this.legacyFixes();

    if( input.pattern )
        try {
            this.settings.pattern = new RegExp(input.pattern);
        } catch(e){}

    if( settings && settings.delimiters ){
        try {
            this.settings.delimiters = new RegExp("[" + settings.delimiters + "]");
        } catch(e){}
    }

    this.id = Math.random().toString(36).substr(2,9), // almost-random ID (because, fuck it)
    this.value = []; // An array holding all the (currently used) tags
    this.DOM = {}; // Store all relevant DOM elements in an Object
    this.extend(this, new this.EventDispatcher());
    this.build(input);
    this.events();
    this.lastSuggestion = ""; //***Added***
}

Tagify.prototype = {
    DEFAULTS : {
        delimiters          : ",",        // [regex] split tags by any of these delimiters
        pattern             : "",         // pattern to validate input by
        callbacks           : {},         // exposed callbacks object to be triggered on certain events
        duplicates          : false,      // flag - allow tuplicate tags
        enforceWhitelist    : false,      // flag - should ONLY use tags allowed in whitelist
        autocomplete        : true,       // flag - show native suggeestions list as you type
        whitelist           : [],         // is this list has any items, then only allow tags from this list
        blacklist           : [],         // a list of non-allowed tags
        maxTags             : Infinity,   // maximum number of tags
        suggestionsMinChars : 2,          // minimum characters to input to see sugegstions list
        maxSuggestions      : 10, // ***Modified***
        suggestionsUrl      : "" // ***Added***
    },

    /**
     * Fixes which require backword support
     */
    legacyFixes : function(){
        var _s = this.settings;

        // For backwards compatibility with older versions, which use 'enforeWhitelist' instead of 'enforceWhitelist'.
        if( _s.hasOwnProperty("enforeWhitelist") && !_s.hasOwnProperty("enforceWhitelist") ){
            _s["enforceWhitelist"] = _s["enforeWhitelist"];
            delete _s["enforeWhitelist"];
            console.warn("Please update your Tagify settings. The 'enforeWhitelist' property is deprecated and you should be using 'enforceWhitelist'.");
        }
    },

    /**
     * builds the HTML of this component
     * @param  {Object} input [DOM element which would be "transformed" into "Tags"]
     */
    build : function( input ){
        var that = this,
            value = input.value,
            inputHTML = '<div><input list="tagifySuggestions'+ this.id +'" class="placeholder"/><span>'+ input.placeholder +'</span></div>';
        this.DOM.originalInput = input;
        this.DOM.scope = document.createElement('tags');
        input.className && (this.DOM.scope.className = input.className); // copy any class names from the original input element to the Tags element
        this.DOM.scope.innerHTML = inputHTML;
        this.DOM.input = this.DOM.scope.querySelector('input');

        if( this.settings.readonly )
            this.DOM.scope.classList.add('readonly')

        input.parentNode.insertBefore(this.DOM.scope, input);
        this.DOM.scope.appendChild(input);

        // if "autocomplete" flag on toggeled & "whitelist" has items, build suggestions list
        if( this.settings.autocomplete && (this.settings.whitelist.length || this.settings.suggestionsUrl != '') ){ // ***Modified***
            if( "suggestions" in this )
                this.suggestions.init();
            else
                this.DOM.datalist = this.buildDataList();
        }

        // if the original input already had any value (tags)
        if( value )
            this.addTags(value).forEach(function(tag){
                tag && tag.classList.add('tagify--noAnim');
            });
    },

    /**
     * Reverts back any changes made by this component
     */
    destroy : function(){
        this.DOM.scope.parentNode.appendChild(this.DOM.originalInput);
        this.DOM.scope.parentNode.removeChild(this.DOM.scope);
    },

    /**
     * Merge two objects into a new one
     */
    extend : function(o, o1, o2){
        if( !(o instanceof Object) ) o = {};

        if( o2 ){
            copy(o, o2)
            copy(o, o1)
        }
        else
            copy(o, o1)

        function copy(a,b){
            // copy o2 to o
            for( var key in b )
                if( b.hasOwnProperty(key) )
                    a[key] = b[key];
        }

        return o;
    },

    /**
     * A constructor for exposing events to the outside
     */
    EventDispatcher : function(){
        // Create a DOM EventTarget object
        var target = document.createTextNode('');

        // Pass EventTarget interface calls to DOM EventTarget object
        this.off = target.removeEventListener.bind(target);
        this.on = target.addEventListener.bind(target);
        this.trigger = function(eventName, data){
            var e;
            if( !eventName ) return;

            if( this.isJQueryPlugin )
                $(this.DOM.originalInput).triggerHandler(eventName, [data])
            else{
                try {
                    e = new CustomEvent(eventName, {"detail":data});
                }
                catch(err){
                    e = document.createEvent("Event");
                    e.initEvent("toggle", false, false);
                }
                target.dispatchEvent(e);
            }
        }
    },

    /**
     * DOM events listeners binding
     */
    events : function(){
        var that = this,
            events = {
                 //  event name / event callback / element to be listening to
                paste   : ['onPaste'      , 'input'],
                focus   : ['onFocusBlur'  , 'input'],
                blur    : ['onFocusBlur'  , 'input'],
                input   : ['onInput'      , 'input'],
                keydown : ['onKeydown'    , 'input'],
                click   : ['onClickScope' , 'scope']
            },
            customList = ['add', 'remove', 'duplicate', 'maxTagsExceed', 'blacklisted', 'notWhitelisted'];

        for( var e in events )
            this.DOM[events[e][1]].addEventListener(e, this.callbacks[events[e][0]].bind(this));

        customList.forEach(function(name){
            that.on(name, that.settings.callbacks[name])
        })

        if( this.isJQueryPlugin )
            $(this.DOM.originalInput).on('tagify.removeAllTags', this.removeAllTags.bind(this))
    },

    /**
     * DOM events callbacks
     */
    callbacks : {
        onFocusBlur : function(e){
            var text =  e.target.value.trim();

            if( e.type == "focus" )
                e.target.className = 'input';
            else if( e.type == "blur" && text ){
                if( this.addTags(text).length )
                    e.target.value = '';
            }
            else{
                e.target.className = 'input placeholder';
                this.DOM.input.removeAttribute('style');
            }
        },

        onKeydown : function(e){
            var s = e.target.value,
                lastTag,
                that = this;

            if( e.key == "Backspace" && (s == "" || s.charCodeAt(0) == 8203) ){
                lastTag = this.DOM.scope.querySelectorAll('tag:not(.tagify--hide)');
                lastTag = lastTag[lastTag.length - 1];
                this.removeTag( lastTag );
            }
            if( e.key == "Escape" ){
                e.target.value = '';
                e.target.blur();
            }
            if( e.key == "Enter" ){
                e.preventDefault(); // solves Chrome bug - http://stackoverflow.com/a/20398191/104380
                if( this.addTags(s).length )
                    e.target.value = '';
                return false;
            }
            else{
                if( this.noneDatalistInput ) clearTimeout(this.noneDatalistInput);
                this.noneDatalistInput = setTimeout(function(){ that.noneDatalistInput = null }, 50);
            }
        },

        onInput : function(e){
            var value = e.target.value,
                lastChar = value[value.length - 1],
                isDatalistInput = !this.noneDatalistInput && value.length > 1,
                showSuggestions = value.length >= this.settings.suggestionsMinChars,
                datalistInDOM;

            e.target.style.width = ((e.target.value.length + 1) * 7) + 'px';


            // if( value.indexOf(',') != -1 || isDatalistInput ){
            if( value.slice().search(this.settings.delimiters) != -1 || isDatalistInput ){
                if( this.addTags(value).length )
                    e.target.value = ''; // clear the input field's value
            }
            else if (this.settings.autocomplete && (this.settings.whitelist.length || this.settings.suggestionsUrl != '') ){ // ***Modified***
                datalistInDOM = this.DOM.input.parentNode.contains( this.DOM.datalist );

                // if sugegstions should be hidden
                if( !showSuggestions && datalistInDOM )
                    this.DOM.input.parentNode.removeChild(this.DOM.datalist)
                else if( showSuggestions ){ // ***Modified***

                    // ***Added/Modified***
                    var suggestion = value.substr(0, this.settings.suggestionsMinChars);

                    if (!datalistInDOM || suggestion !== this.lastSuggestion) {
                        this.lastSuggestion = suggestion;

                        var dom = this.DOM;

                        if (this.settings.suggestionsUrl != '') {
                            if (datalistInDOM) dom.input.parentNode.removeChild(dom.datalist);
                            dom.input.parentNode.appendChild(dom.datalist);
                            $(dom.datalist).find('select option').remove();

                            $.getJSON(this.settings.suggestionsUrl + this.lastSuggestion, function (data) {
                                $.each(data, function (index, item) {
                                    $(dom.datalist).find('select').append('<option>' + item + '</option>');
                                });
                            });
                        }
                    }
                    // ***End of Added***

                }
            }
        },

        onPaste : function(e){
            var that = this;
            if( this.noneDatalistInput ) clearTimeout(this.noneDatalistInput);
            this.noneDatalistInput = setTimeout(function(){ that.noneDatalistInput = null }, 50);
        },

        onClickScope : function(e){
            if( e.target.tagName == "TAGS" )
                this.DOM.input.focus();
            if( e.target.tagName == "X" ){
                this.removeTag( e.target.parentNode );
            }
        }
    },

    /**
     * Build tags suggestions using HTML datalist
     * @return {[type]} [description]
     */
    buildDataList : function(){
        var OPTIONS = "",
            i,
            datalist = document.createElement('datalist');

        datalist.id = 'tagifySuggestions' + this.id;
        datalist.innerHTML = "<label> 
                                select from the list: 
                                <select> 
                                    <option value=''></option> 
                                    [OPTIONS] 
                                </select> 
                            </label>";

        if (this.settings.whitelist.length) { // ***Added***
            for( i=this.settings.whitelist.length; i--; )
                OPTIONS += "<option>"+ this.settings.whitelist[i] +"</option>";
        } // ***Added***

        datalist.innerHTML = datalist.innerHTML.replace('[OPTIONS]', OPTIONS); // inject the options string in the right place

      //  this.DOM.input.insertAdjacentHTML('afterend', datalist); // append the datalist HTML string in the Tags

        return datalist;
    },

    getNodeIndex : function( node ){
        var index = 0;
        while( (node = node.previousSibling) )
            if (node.nodeType != 3 || !/^s*$/.test(node.data))
                index++;
        return index;
    },

    /**
     * Searches if any tag with a certain value already exis
     * @param  {String} s [text value to search for]
     * @return {boolean}  [found / not found]
     */
    isTagDuplicate : function(s){
        return this.value.some(function(item){ return s.toLowerCase() === item.value.toLowerCase() });
    },

    /**
     * Mark a tag element by its value
     * @param  {String / Number} value  [text value to search for]
     * @param  {Object}          tagElm [a specific "tag" element to compare to the other tag elements siblings]
     * @return {boolean}                [found / not found]
     */
    markTagByValue : function(value, tagElm){
        var tagsElms, tagsElmsLen;

        if( !tagElm ){
            tagsElms = this.DOM.scope.querySelectorAll('tag');
            for( tagsElmsLen = tagsElms.length; tagsElmsLen--; ){
                if( tagsElms[tagsElmsLen].textContent.toLowerCase().includes(value.toLowerCase()) )
                    tagElm = tagsElms[tagsElmsLen];
            }
        }

        // check AGAIN if "tagElm" is defined
        if( tagElm ){
            tagElm.classList.add('tagify--mark');
            setTimeout(function(){ tagElm.classList.remove('tagify--mark') }, 2000);
            return true;
        }

        else{

        }

        return false;
    },

    /**
     * make sure the tag, or words in it, is not in the blacklist
     */
    isTagBlacklisted : function(v){
        v = v.split(' ');
        return this.settings.blacklist.filter(function(x){ return v.indexOf(x) != -1 }).length;
    },

    /**
     * make sure the tag, or words in it, is not in the blacklist
     */
    isTagWhitelisted : function(v){
        return this.settings.whitelist.indexOf(v) != -1;
    },

    /**
     * add a "tag" element to the "tags" component
     * @param  {String/Array} tagsItems [A string (single or multiple values with a delimiter), or an Array of Objects]
     * @return {Array} Array of DOM elements (tags)
     */
    addTags : function( tagsItems ){
        var that = this,
            tagElems = [];

        this.DOM.input.removeAttribute('style');

        /**
         * pre-proccess the tagsItems, which can be a complex tagsItems like an Array of Objects or a string comprised of multiple words
         * so each item should be iterated on and a tag created for.
         * @return {Array} [Array of Objects]
         */
        function normalizeTags(tagsItems){
            var whitelistWithProps = this.settings.whitelist[0] instanceof Object,
                isComplex = tagsItems instanceof Array && "value" in tagsItems[0], // checks if the value is a "complex" which means an Array of Objects, each object is a tag
                result = tagsItems; // the returned result

            // no need to continue if "tagsItems" is an Array of Objects
            if( isComplex )
                return result;

            // search if the tag exists in the whitelist as an Object (has props), to be able to use its properties
            if( !isComplex && typeof tagsItems == "string" && whitelistWithProps ){
                var matchObj = this.settings.whitelist.filter(function(item){
                    return item.value.toLowerCase() == tagsItems.toLowerCase();
                })

                if( matchObj[0] ){
                    isComplex = true;
                    result = matchObj; // set the Array (with the found Object) as the new value
                }
            }

            // if the value is a "simple" String, ex: "aaa, bbb, ccc"
            if( !isComplex ){
                tagsItems = tagsItems.trim();
                if( !tagsItems ) return [];

                // go over each tag and add it (if there were multiple ones)
                result = tagsItems.split(this.settings.delimiters).map(function(v){
                    return { value:v.trim() }
                });
            }

            return result.filter(function(n){ return n }); // cleanup the array from "undefined", "false" or empty items;
        }

        /**
         * validate a tag object BEFORE the actual tag will be created & appeneded
         * @param  {Object} tagData  [{"value":"text", "class":whatever", ...}]
         * @return {Boolean/String}  ["true" if validation has passed, String or "false" for any type of error]
         */
        function validateTag( tagData ){
            var value = tagData.value.trim(),
                maxTagsExceed = this.value.length >= this.settings.maxTags,
                isDuplicate,
                eventName__error,
                tagAllowed;

            // check for empty value
            if( !value )
                return "empty";

            // check if pattern should be used and if so, use it to test the value
            if( this.settings.pattern && !(this.settings.pattern.test(value)) )
                return "pattern";

            // check if the tag already exists
            if( this.isTagDuplicate(value) ){
                this.trigger('duplicate', value);

                if( !this.settings.duplicates ){
                    // this.markTagByValue(value, tagElm)
                    return "duplicate";
                }
            }

            // check if the tag is allowed by the rules set
            tagAllowed = !this.isTagBlacklisted(value) && (!this.settings.enforceWhitelist || this.isTagWhitelisted(value)) && !maxTagsExceed;

            // Check against blacklist & whitelist (if enforced)
            if( !tagAllowed ){
                tagData.class = tagData.class ? tagData.class + " tagify--notAllowed" : "tagify--notAllowed";

                // broadcast why the tag was not allowed
                if( maxTagsExceed )                                                        eventName__error = 'maxTagsExceed';
                else if( this.isTagBlacklisted(value) )                                    eventName__error = 'blacklisted';
                else if( this.settings.enforceWhitelist && !this.isTagWhitelisted(value) ) eventName__error = 'notWhitelisted';

                this.trigger(eventName__error, {value:value, index:this.value.length});

                return "notAllowed";
            }

            return true;
        }

        /**
         * appened (validated) tag to the component's DOM scope
         * @return {[type]} [description]
         */
        function appendTag(tagElm){
            this.DOM.scope.insertBefore(tagElm, this.DOM.input.parentNode);
        }

        //////////////////////
        tagsItems = normalizeTags.call(this, tagsItems);

        tagsItems.forEach(function(tagData){
            var isTagValidated = validateTag.call(that, tagData);

            if( isTagValidated === true || isTagValidated == "notAllowed" ){
                // create the tag element
                var tagElm = that.createTagElem(tagData);

                // add the tag to the component's DOM
                appendTag.call(that, tagElm);

                // remove the tag "slowly"
                if( isTagValidated == "notAllowed" ){
                    setTimeout(function(){ that.removeTag(tagElm, true) }, 1000);
                }

                else{
                    // update state
                    that.value.push(tagData);
                    that.update();
                    that.trigger('add', that.extend({}, tagData, {index:that.value.length, tag:tagElm}));

                    tagElems.push(tagElm);
                }
            }
        })

        return tagElems
    },

    /**
     * creates a DOM tag element and injects it into the component (this.DOM.scope)
     * @param  Object}  tagData [text value & properties for the created tag]
     * @return {Object} [DOM element]
     */
    createTagElem : function(tagData){
        var tagElm = document.createElement('tag');

        // for a certain Tag element, add attributes.
        function addTagAttrs(tagElm, tagData){
            var i, keys = Object.keys(tagData);
            for( i=keys.length; i--; ){
                var propName = keys[i];
                if( !tagData.hasOwnProperty(propName) ) return;
                tagElm.setAttribute(propName, tagData[propName] );
            }
        }

        // The space below is important - http://stackoverflow.com/a/19668740/104380
        tagElm.innerHTML = "<x></x><div><span title='"+ tagData.value +"'>"+ tagData.value +" </span></div>";

        // add any attribuets, if exists
        addTagAttrs(tagElm, tagData);

        return tagElm;
    },

    /**
     * Removes a tag
     * @param  {Object}  tagElm    [DOM element]
     * @param  {Boolean} silent    [A flag, which when turned on, does not removes any value and does not update the original input value but simply removes the tag from tagify]
     */
    removeTag : function( tagElm, silent ){
        var tagData,
            tagIdx = this.getNodeIndex(tagElm);

        if( !tagElm) return;

        tagElm.style.width = parseFloat(window.getComputedStyle(tagElm).width) + 'px';
        document.body.clientTop; // force repaint for the width to take affect before the "hide" class below
        tagElm.classList.add('tagify--hide');

        // manual timeout (hack, since transitionend cannot be used because of hover)
        setTimeout(function(){
            tagElm.parentNode.removeChild(tagElm);
        }, 400);

        if( !silent ){
            tagData = this.value.splice(tagIdx, 1)[0]; // remove the tag from the data object
            this.update(); // update the original input with the current value
            this.trigger('remove', this.extend({}, tagData, {index:tagIdx, tag:tagElm}));
        }
    },

    removeAllTags : function(){
        this.value = [];
        this.update();
        Array.prototype.slice.call(this.DOM.scope.querySelectorAll('tag')).forEach(function(elm){
            elm.parentNode.removeChild(elm);
        });
    },

    /**
     * update the origianl (hidden) input field's value
     */
    update : function(){
        var tagsAsString = this.value.map(function(v){ return v.value }).join(',');
        this.DOM.originalInput.value = tagsAsString;
    }
}

})(jQuery);

10 things a web developer should be careful of

2018-04-23T12:00:00+10:00

I am a bit old-fashioned when it comes to web development. As much as I like the fancy new ways of doing things, I see so many websites where things go wrong and end up frustrating users.

Viewing websites on mobile phones is often the most frustrating, particularly responsive websites where the same website is designed to run on both a "resource-limited" mobile as well as a "resource-rich" desktop.

Here's my list of ten things web developers should be careful of.

Browser detection
I have a Windows phone (yes, I am the one) and there's one company that sends me a regular email with an offer that sometimes I want to activate so I click the link in the email on my phone and it takes me to a web page. The "clever" developers have decided that because they don't recognise my browser's user agent that they will just show me an error message that my browser is not compatible. The offer doesn't get activated and I can't view any other pages on that website either. They have decided that their precious website can only be used if they know what browser is in use, just so it can display a message saying an offer has been accepted.
In this case it would be much better to at least perform the server-side action of activating the offer, and attempt to display a success message.
Here's another case of browser detection gone too far. A front end developer creating a new website for a large organisation decided that he should display a (non-dismissable) message across the top of every page of the website if he detected that the user was using an old browser that wasn't fully compatible with whatever HTML/CSS features he had used. Something like this...
"Your browser is out of date. Please visit xxx to update your browser."
Nice. Unfortunately the several thousand staff in this organisation were all locked into using a specific browser that just happened to be old enough that they would all see this message and they were not actually able to update their browser due to organisational policy.
So please don't tell people what they probably already know. Chances are users know they are not operating on the latest browser, and are either unable to update it for some reason or don't want to update it.
Make your website accessible
Many web developers know just the basics of web accessibility and nothing more.
My advice to every web developer is to actually try a screen reader and other assistive technology yourself and see what it's like to navigate a web page.
You'll soon learn how frustrating it must be for people to use web pages even if the website is properly accessible. And how infuriating it must be if it isn't done well.
You'll learn the importance of properly structured navigation, content and why often a blank image "alt" tag is better than your 50 word description of some image that you just used to make the page look nice. And you'll find out all about the joys of HTML tables and accessibility.
Be careful about what you do on page load
Don't you hate it when you go to a web page, one that is a little slow to load fully, and you start reading down the page, then some clever web developer decides that the page has now fully loaded and you should be scrolled to a specific part of the page, or you start typing into a text box and suddenly the focus is given to a different control?
Be careful with what and when you do these types of things.
AJAX is great, but...
AJAX has some great advantages and can make pages operate quicker and more like "desktop" applications, but you have to get them right.
Here's some things that can go wrong:
History: many people rely on the back button. If you have some nice infinity load list and someone clicks through to one of the items on the list and then clicks back, what happens? Are they taken back to the exact same list they just saw? Or are they taken back to the top and then have to infinity load back to where they were? Many websites don't get this right (because it can be difficult).
Bookmarks: On a list page, maybe there's a bunch of filters and I've set them up with how I want to see the list and bookmarked it so I can always come back to that list. Imagine my surprise when they filters are all AJAX and the URL is not updated so I just bookmark the standard page instead!
Allow open in new window/tab
Some websites accidentally block opening of hyperlinks in new windows/tabs. Middle-clicking the hyperlink just opens in the same window/tab, and/or the actually hyperlink is not a "proper" html hyperlink, just some other element with some JavaScript to make it work so right-clicking doesn't bring up a hyperlink-appropriate context menu.
Some people (like me) often like to open a heap of items from a list page at the same time without having to keep going back to the list.
Speed
Google know how frustrating slow websites can be, so they rank slow websites below similar faster websites. They even back AMP (Accelerated Mobile Pages) in an attempt to speed the web up (I don't like AMP very much though).
I think that one of the biggest threats to performance is too much JavaScript. These days many web developers seem dependent on front-end frameworks that have a heap of JavaScript plus dependencies on other frameworks and libraries that also have a heap of JavaScript.
I'm guilty of this too with my preference towards Bootstrap and jQuery (and their dependencies). Part of this is just laziness. See youmightnotneedjquery.com for how simple it is to do most things without jQuery.
But advertising scripts are probably the worst offenders. We've "advanced" from simple banner and text advertising to complex dynamic advertisements that scroll through pages of items you previously looked at on some other website.
Times have changed, people are used to seeing more on the page
In the early days of the web, pages with lists - such as products on an ecommerce website - used to be short, like maybe ten products per page.
These days people are used to seeing a lot more items per list before they have to load the next page, or even be able to see everything perhaps with infinity scroll.
I still see many recent websites that have ridiculously small lists that requires a lot of pagination to get through a long list, and an equally frustrating trip through browser history to get back to that other page you want to have a second look at.
As per tip 4, using AJAX for lists pagination or infinity scrolling needs to be done very carefully otherwise it can be equally frustrating. The middle ground is perhaps non-AJAX pagination but with a generous amount of items per page.
Don't hide too much on mobile
Some websites have separate mobile versions or are responsive and hide a lot of features that are available on a desktop.
This can be very frustrating if you know about that feature from using on a desktop previously but can't find it when viewing on a mobile.
I'm all for hiding superfluous content on the mobile, sometimes even things like breadcrumbs that can take up valuable real estate at the top of the page, but generally every feature should be accessible and available on all device versions.
Test properly
Often web developers have high-end mobile phones and desktops and do a lot of testing in their own high-speed development environment.
You won't really know how well your website works in the "real world" unless you test in a similar environment to what your users will be doing. That means possibly a low-end phone, wi-fi or 3G, away from your local server network, and on a typically loaded server.
Don't override normal behaviour
I recently visited a website that had an autocomplete type search.
The problem was that as I was typing, it was performing an AJAX query, and while it was waiting for the server response any keystrokes were ignored.
I typed "the quick brown fox jumps over the lazy dog" and it showed as "the quck brown fox umps ve the lay dog". And I'm not even a fast typist.
Best to let the browser do it's job here, and just listen in to what's being typed without intercepting it.

Why Microsoft's ASP/ASP.NET may be the safe choice for development

2018-02-15T12:00:00+10:00

Your situation may be different but in my case, over close to 20 years I have created many websites and applications for myself and clients, many that are still in use today.

Most, if not all, have under gone some changes over the years. In some cases this might just be a cosmetic refresh and the server-side code remains the same or features may have been added or changed over the years.

Of course, due to the nature of the web, some get total rewrites after a short time, but many clients don’t need or want to spend money on major rewrites.

Fortunately, the majority of the websites and applications I have developed were done so using the Microsoft technologies Classic ASP and ASP.NET.

If, like me, you created something in the late 1990s using Classic ASP and didn’t do anything stupid like not protect against SQL injection or unsafe user uploads, then your code could still run today on a modern web server running a current version of operating system and IIS without you losing sleep that there were security problems waiting to bite you.

And if, like me, you wrote something in ASP.NET 1.x or 2.x in the early 2000s, again, as long as it was running on a modern server, you could feel safe and without fear of potential security problems.

Now, let’s compare this to PHP (and let me say I am no expert on PHP but do have some experience with it).

If you wrote something in an earlier version of PHP, because there are breaking changes between major versions, you either still need to be running on that same version of PHP, or it’s time to review and migrate to the current version of PHP.

An online search reveals that some earlier versions of PHP have security flaws and the advice is to update to safe versions.

What about these popular open source frameworks that are so popular now?

These days it’s a very common for developers to use new frameworks, usually open source, to develop their websites and applications. There seems to be a new one out every month or so.

Is this a good idea to use these? Will an application written using one of these frameworks still be okay in a few years’ time?

Here’s a hypothetical look at what the future might hold for such an application…

1. Our application is developed in version 2 of a fictional JavaScript framework. Version 1 was well received but had some security and performance issues which have been resolved by a major rewrite. The fictional JavaScript framework leverages off a couple of other JavaScript frameworks.

2. Our application works great and gets a few enhancements over the next year or two.

3. Version 3 of the fictional JavaScript framework gets introduced and has some great new features and fixes some security and performance issues with version 2. There are some breaking changes from the previous version so our application code will need to be reviewed and migrated.

4. Our application is quite large, possibly with hundreds of thousands of lines of code, and reviewing and migration is a major project that we would rather not undertake.

5. Interest and support for version 2 of the fictional framework has dropped off and as version 3 was not well received another fictional framework has become the popular choice. It is strongly recommended not to use version 2 anymore because of security flaws in it and/or one of its dependencies. Because of this, and your development team are excited about using the new fictional framework, a rewrite is recommend.

Did I mention I still have Classic ASP and early .NET Web Forms applications out there in the wild and I don’t lose any sleep over them at all?

The command line interface is back

2017-04-28T12:00:00+10:00

Back in the 1980s and early 1990s I considered myself a bit of an expert at the MS-DOS command line interface and knew all the commands and all of the options for each command.

Then Windows 3.x came along and things started moving away from the command line until Windows 95, when the command line was no longer needed by the average user. Everything could be done with a GUI.

You could then administer most functions of the computer, even set up the operating system without the need to type in any commands.

I was a late adopter of Windows, but became comfortable in a GUI world and was happy to be free of the command line interface.

But slowly things have been regressing back to the command line interface.

I blame Linux. Even today, some familiarity with the command line is essential for using a Linux operating system.

But unfortunately, the return of the command line interface is not just limited to using a Linux OS.

As a web developer, I am now inundated with various command line interfaces, some of which offer no GUI alternative. I appreciate that most people in this profession are most likely to be advanced computer users and not afraid of command line interfaces, but for how many purposes is a CLI really better than a GUI?

Source control systems, tools for installing add-ons and dependencies, and even configuring some applications now often requires typing of commands and editing of text-based configuration files.

Can we go back to the future now, please?

Is ASP.NET better than PHP?

2016-09-21T12:00:00+10:00

Firstly, I know PHP is a programming language and ASP.NET is a framework so we're not comparing apples with apples here, but still people make the choice whether they are going to learn how to develop in PHP or ASP.NET so this comparison is valid!

This posts stems from my own curiosity. It's a question not a statement. I've been developing in ASP.NET for many years and I like it. I've dabbled a bit in PHP and there are some things I like and some I don't like, but maybe I haven't immersed myself in it enough to know what's great about it...

So here's what I think is great about ASP.NET. Can you tell me how things are from the PHP side?

Visual Studio

(Microsoft's IDE for ASP.NET development)

Visual Studio is awesome. Why? For a start it autocompletes commands, objects, methods, properties and almost everything else as I type and it has an extra level of intelligence that almost can't be explained. If I do something like object1.Field = object2.Field and both objects have the same property name then it gives that property as the first choice after I type the part object2.. And as I start typing something it gives the last similar name as the first choice, which is useful when you are writing code and repetitively typing the same object names.

VS will tell me when there's an error - a red wavy underline tells you something's seriously wrong - and it will tell me if something doesn't make sense - a green wavy underline might show me something like code that's unreachable, or accessing a value where all possible code paths don't set a value.

It also has great navigability. What do I mean? Moving around your code is easy. Recently worked on areas are highlighted. You can jump to a line number easily. Ctrl+c with no selection copies the current line, and pasting a whole line with CTRL+v doesn't require you to pin-point an exact location - VS will put it above the current line if your cursor is in the middle of another line. Correct indenting happens automatically when it should.

I can do all sorts of things with code windows and toolbars... Dock them, split the screen, move them to other screens!

This is only just scratching the surface of VS's features, and all of this power comes at a price of as little as $zero (there's nothing wrong with the free version either).

Debugging

Also a feature of Visual Studio and it's integrated web server is debugging. I can watch as each line executes and check values. I can set breakpoints and watches.

Compilation

You don't have to compile all ASP.NET projects, but it is very useful to do so.

You can't compile a project if there are errors in the coding. Most markup errors will not show up in compilation but coding errors will result in a failed build.

Better to know at compile time than after copying changes to a test or live website, right?

Choice of development style

Just like PHP and Classic ASP you can code top-to-bottom style using ASP.NET "Web Pages". Although aimed at beginners and PHP converts, you have full access to the power of C# and the .NET framework. You don't need to compile, you can just copy files to your web folder and they will be compiled automatically as needed.

Prefer the more modern and trendy MVC? You can do that too.

Or what about every anti-Microsoft-er's favourite: "Web Forms". Hated almost everywhere for its abstraction from everything (shouldn't we all be writing assembly code then?), inefficient Viewstate (you know you can turn it off right?) and lack of compliance with modern standards (Microsoft could have updated it...).

My opinion on "Web Forms"? It's a work of genius! But like most systems it's simple to make simple applications, but difficult to make difficult applications. You can do a lot wrong with it if you don't have experience with it... Just like any other form of development.

If you have to make something like a real complicated web form with lots of AJAX for hiding and showing things then "Web Forms" and UpdatePanels are hard to beat. But I'm sure you won't believe me. I've got one such horrendously complex form that has 2,400 lines of markup and another 7,850 lines of code-behind - I don't know how I could have done that in any other system (I'm bracing myself for the "why would you" comments now).

Strongly typed

I know why some people don't like strongly typed objects but it makes things so much easier. I don't know how people can cope without it? All this checking for this and that, and then can it be converted to this, okay then convert it. It drives me crazy when I have to switch back to Classic ASP and do all that.

Linq to SQL

Recently I was wanting to go back to writing plain SQL queries so I could write the absolute most efficient data access for a performance critical project (only because it was going to be run on cheap shared hosting). I didn't last through more than a couple of pages before I realised that doing things this way seems ridiculous in 2016.

Linq to SQL (or Linq to x), despite its faults, is the only way I want to query databases now.

If you don't know about it, it allows you to write database queries within your code that are similar to SQL. You can pretty much do most things in L2S as you can in SQL, and the syntax is similar (but admittedly some things need a bit of extra work). You can also keep adding to your queries before they actually go to the database server (eg. if you need to build a complex query based on conditions).

The great thing about L2S is that your database structure is known so your queries are strongly typed. You can't mistype a table or column name.

I know there's ORMs and the like out there which will easily generate simple queries for you without having to write your own SQL queries but, I don't know about you, but most of my SQL queries are not just simple "SELECT WHERE" type queries. I usually have all sorts of joins and conditions and counts and other stuff going on.

Now if only Microsoft would make Linq to SQL even more like SQL (close to identical would be nice) then I would be really happy.

It's in the framework

Almost everything you need is already baked into the framework. Image manipulation, email, database access, even authentication and authorisation if you like. It can be rare to have to go outside the framework for most tasks. I know choice is a great thing but sometimes not having to make a choice is better.

So how do these things compare to PHP? What do you like in PHP that makes development better than ASP.NET?

2023 Update

My reaction to reading this post seven years later?

I moved from ASP.NET to PHP in my wok and now have a different perspective.

Firstly, PHP Storm in an excellent IDE for PHP and comparable to Visual Studio except there is no free edition. You can do everything that I mentioned VS can do.

.NET has evolved, and not everything is good. All code must now be compiled. The framework no longer has everything you need. At least MVC is no longer being pushed as the answer to everything. Linq is still awesome for writing queries.

PHP has also evolved. It appears to be slowly becoming strongly-typed... Well, kind of. I don't know why they don't just go all-out and introduce a configuration option and give people the option of what they want.

I hope Accelerated Mobile Pages (AMP) fail

2016-05-23T12:00:00+10:00

It's not often I wish such bad fortune on a new innovation but I am wishing Accelerated Mobile Pages (AMP) dies a quick death.

It's not that I don't agree that websites should be MUCH more efficient and load quicker.

It's just that I don't agree that this is the way to make it happen.

If you don't already know about AMP, in 25 words or less it's a way of creating web pages that are small and faster loading. AMP pages cannot contain certain HTML (like forms), only include images using a special AMP tag, only allow approved scripts, don't allow external style sheets, and the total page size must be less than 50kb.

The most important thing you need to know about AMP is that Google are backing it, and "supporting" it by making it a ranking factor.

Not that long in the past many of us developers were creating websites that were optimised for a specific resolution, usually a maximum width of 1024 pixels so targeted at desktop screens.

Then mobile browsing became popular and many of us started creating mobile only websites. These were usually cut down versions of the desktop site and were faster less bandwidth hungry (and usually uglier) but meant that we now had to maintain two websites.

Then along came "responsive design" and we were able to create one website for all devices.

Now Google wants us to create AMP versions of all our web pages.

I'm not sure the world wants cut down versions of web pages. As a frequent mobile browser (someone who browses, not something like Internet Explorer) myself, there are times when I appreciate a simple, easy to read alternate mobile only version of a web page, but there are also many times when I can't find some relevant feature on a mobile page that I am used to seeing on the equivalent desktop page. Often enough, I find myself looking for a "view full site" hyperlink.

Given the high rate of technical advances in mobile technology, and the ongoing reduction in cost of mobile data, I hope that AMP might not be relevant.

One-way Web?

As I said earlier in this post, AMP doesn't allow HTML forms. This might be okay for some types of websites, but many websites (possibly most) have a real purpose for forms such as posting a comment on a blog or news website, subscribing to a newsletter, or buying something on an ecommerce website. How do you accommodate this? Move these things to separate pages, and then you would probably want to style them the same as the AMP pages which means you are now maintaining a desktop version of the site, AMP version of the site, and these AMP spin-off pages.

Is your website supported by advertising? Well, only certain advertising platforms' scripts are approved, so hopefully you use one of them and not one of the unsupported platforms. Google AdWords is supported, of course.

Disallowing unapproved scripts mean that you won't be able to add any AJAX either. AMP pretty much makes the web one-way. How 1990s!

So, if I am against AMP, what could Google do instead?

I think they already had some good things going like suggestions for optimisations and improvements in Google Search Console. Telling people what's wrong with their websites and how to fix them is a good step.

Punishing us for producing slow websites, as much as I won't like being a victim, is also probably the right thing to do.

But how about if they got involved in some of these open source systems that are used by so many websites and improving them? I am talking about open source applications such as CMS and eCommerce, many of which are horribly inefficient. And popular open source frameworks like Bootstrap and jQuery.

Web developers: Don't feel too bad if your websites have a few bugs

2016-03-21T12:00:00+10:00

I know there are bugs in most (all?) of the websites I have developed, and I feel bad when I find a bug and wonder how long it's been there.

But no matter how many bugs there are in your code, there's probably a worse website... Yahoo!

It amazes me how a large company like Yahoo - with all their resources and no doubt a large team of programmers - can have so many serious bugs in their website.

I recently wanted to investigate Yahoo's Gemini advertising service, as I am getting a bit sick of Google AdWords and it's ridiculous number of resources that need to load just to show one advertisement.

First issue... I have an old Yahoo account, and I can log in to this account, but it seems to be an old account that doesn't allow me to make any changes to update my account, just an error message"Yahoo Profile (profile.yahoo.com) is no longer active" and a link to learn more about the error that doesn't really help at all.

Okay, no problem, I'll create a new account. Second issue... The Yahoo CAPTCHA is showing a mixture of upper and lower case so I enter the characters exactly as I see them. It takes me a few failed attempts until I try all lower case, even though they show as upper case, and I finally succeed.

Let's check my account settings and set my preferences. From the Yahoo home page I click my name in the top right corner which brings up a drop down menu of choices. Third issue... As soon as my mouse leaves my name the drop down disappears. It seems impossible to select anything from this menu! Why have a click-to-activate drop down menu that disappears on hover?

I move on to Yahoo's Gemini website and attempt to sign up. At this point I am using my Windows Phone 8.1 and trying to select my country but - fourth issue - Yahoo are using their own SELECT replacement drop down menus which do not let me scroll through the choices, so although I want to select Australia, I am stuck down at the end of the alphabet because United States is the default value.

I switch to the desktop and I can now select my country but when I attempt to select my time zone I see - fifth issue - that there are duplicate time zones in the list with no obvious difference. Oh well, choose anyone I guess.

Ready to sign up now? Nope, sixth issue, I see a message saying that only US based companies are supported at this time. Why not show this message before I fill out the form? Why allow me to select from several hundred countries if only one is supported?

Time to give up on Yahoo!

Another website project for motoring magazine collectors

2016-02-03T12:00:00+10:00

I'm working on a new website for one of my hobbies – car magazine collecting.

I wanted a website where I could create an index of the contents of each magazine so I would be able to search for articles about particular makes and models.

I also wanted to be able to track which issues I have, which ones I want, and which ones I don't want and would like to sell.

A wiki-like website where anyone could add and update magazine contents sounded ideal.

I love stackoverflow.com and have taken some inspiration from that site for how to handle members, permissions and points.

So, what to develop in?

Well I work in ASP.NET Web Forms in my day job but I do get a bit annoyed by some of its features – mainly the bloat and issues with ViewState. It would have been the quickest option as I am so familiar with it.

I thought about ASP.NET MVC and I really should take a leap into this as it seems it is now the professional choice for ASP.NET development. However, I don't like it.

I decided upon ASP.NET Web Pages. One of the main reasons is that I can do development changes and fixes easily without the need for compilation and redeployment. Just a change to a cshtml file is all that's needed.

Although Web Pages is regarded as a framework for beginners, it does expose all the benefits of ASP.NET and C#, so it is just as capable as any of the other choices.

On the client side I have chosen to start with Bootstrap 4. Although still in Alpha stage I thought it would be a good idea to get acquainted with it, and enjoy the benefits of smaller file size, improvements and new features.

And what about data access?

Because this (not profitable) website would be run on a low-cost shared hosting plan, performance would be fairly critical. My original plan for data access was to use the SqlClient namespace and SqlDataReaders, etc for everything. After all, that's similar to how we did things in the "old days" - raw SQL and straight from database to web page.

However, it didn't take long for me to tire of doing it this way. I prefer writing raw SQL but years of Linq to SQL and Entity Framework made me yearn for strongly typed queries and results.

So I started switching over the Linq to SQL. A lot less bloat than EF but it really speeds development. Some of my more complex queries are a bit of a pain in Linq but that's the price you pay. Still, I hope someone answers my call for a better system that integrates application and database, and allows us to write raw SQL in our application better than Linq does.

magazinecollector.net is online but still a work in progress.