How to make better looking webpages on an Arduino webserver?

Question

I have a Teensy 4.1 with ethernet, but I have other networked MCUs like a D1 mini, various ESP8266s and Arduino ethernet shields.

My question is: how I can make better looking web pages on these devices, where having a filesystem is limited by memory, or which lack a filesystem.

Specifically, I would like to add SVG images; I'm interested in how to make things like background images. I understand HTML/CSS, and I can code in C, get my data, whatever, but I want to make the page look good too.

Here's an example of the issue as it relates to HTML being rendered; here is an HTML page:

<!DOCTYPE html>
<html>
  <head> 
    <style>
      body {
           background-image: url(example.svg);
           }
    </style>
  </head>
  <body>
  </body>
</html>

Cool, well to write that in something an Arduino can render, you use print statements; you might write:

// listen for incoming clients
// forgoing all of server setup
EthernetClient client = server.available();
client.println("<!DOCTYPE html><head><style>");
client.println("body {background-image: url(example.svg);}");
client.println("</style></head><body></body></html>");

This won't work without a filesystem, or won't work as is.

Then, I tried inline SVG:

// listen for incoming clients
// forgoing all of server setup
EthernetClient client = server.available();
client.println("<!DOCTYPE html><head><style>");
client.println("body { ");
client.println("background-image: url("data:image/svg+xml;utf8,<svg>**Omited**</svg>);");
client.println("}");
client.println("</style></head><body></body></html>");

This works, sometimes, but seems to have limitations as well.

I'm just wondering how I can make great looking webpages with stylesheets on an Arduino, only using print statements, or not requiring a filesystem.

score 2 · Accepted Answer · answered Dec 10 '21 at 06:31

Your method of embedding SVG seems unnecessarily complicated to me. You don't need to give a URL of anything. Below is valid for putting SVG onto a web page:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html;charset=utf-8" >
  <title>Test</title>
</head>
<body>
<div style="position:absolute; z-index:-1;">
  <svg width="500" height="500">
    <rect x="5" y="5" width="450" height="450" fill="lightblue" stroke="black"></rect>
  </svg>
</div>
<p style="padding:10px;">
Hello, world
</body>
</html>

The "position:absolute" and the z-index styles put the SVG in a fixed position on the page (like a background image) and underneath anything else you might put there.

score 1 · Answer 2 · answered Nov 27 '21 at 18:00

well, I still want to know more about writing great looking webpages on ardunio network devices.

I got my immediate issue sorted for now. html/css seems to have some issues I dont quite understand as they relate to print statements, and Im pretty sure the url attribute is what is getting me.

I found this great github project that encodes svg for css inline functions. https://yoksel.github.io/url-encoder/

I set up a basic web server example to simplify the code I was looking at. its weird, on html pages I have put full style sheets on one line and never had a problem. here, it wouldn't work until I broke the style sheet into multiple lines (even with the url encoded svg).

Here is my simpler example for a page with a background image:

#include <SPI.h>
#include <NativeEthernet.h>
void  pageWrite(EthernetClient client);
void listenClient(EthernetClient client);
byte mac[] = {
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED
};
IPAddress ip(10, 1, 0, 177);
EthernetServer server(80);
void setup() {
delay(5000);  //you want this delay. reason: tl;dr
  Serial.begin(9600); 
  Ethernet.begin(mac, ip);
if (Ethernet.linkStatus() == LinkOFF) {
    Serial.println("Ethernet cable is not connected.");
  }
  // start the server
  server.begin();
  delay(100);
  Serial.print("server is at ");
  Serial.println(Ethernet.localIP());
} //end setup
void loop() {
      EthernetClient client = server.available();
listenClient(client);
}  //end loop

void listenClient(EthernetClient client){
if (client) {
    boolean currentLineIsBlank = true;
    while (client.connected()) {
       if (client.available()) {
          char c = client.read();
       // Serial.write(c);   //tells about the client connection
        if (c == '\n' && currentLineIsBlank) {
          pageWrite(client);
          break;
        }
        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        } else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }//end avail 
    } //end conect
    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
  } //end client
}//end listenClient()
void  pageWrite(EthernetClient client){
client.println("<!DOCTYPE html><html><head><style>");
client.println("body {");
client.println("background-image:"); 
client.println("url(&quot;data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 227.35277 156.29323' height='6.1532764in' width='8.9508963in' %3E%3Cpath style='fill:%23ff0000; stroke:%23ff0000; stroke-width:1.32300019;stroke-miterlimit:4; stroke-dasharray:none; stroke-opacity:1' d='m 12.000819,0.66145838 c -6.2819637,0 -11.33936062,5.05739692 -11.33936062,11.33936062 V 144.29249 c 0,6.28197 5.05739692,11.33936 11.33936062,11.33936 H 215.35224 c 6.28197,0 11.33885,-5.05739 11.33885,-11.33936 V 12.000819 c 0,-6.2819637 -5.05688,-11.33936062 -11.33885,-11.33936062 z M 18.932696,12.696383 H 205.5854 c 5.02557,0 9.07128,4.04571 9.07128,9.071282 v 77.03923 c 0,5.025575 -4.04571,9.071285 -9.07128,9.071285 H 18.932696 c -5.025572,0 -9.0712819,-4.04571 -9.0712819,-9.071285 v -77.03923 c 0,-5.025572 4.0457099,-9.071282 9.0712819,-9.071282 z' /%3E%3Cellipse style='fill:%23ffffff;fill-opacity:1; stroke:%23ff0000; stroke-width:1.32300007; stroke-miterlimit:4; stroke-dasharray:none; stroke-opacity:1' cx='21.16666' cy='130.59085' rx='10.583334' ry='9.0714283' /%3E%3Cellipse style='fill:%23ffffff; fill-opacity:1; stroke:%23ff0000; stroke-width:1.32300007; stroke-miterlimit:4; stroke-dasharray:none;stroke-opacity:1' cx='200.62982' cy='131.57361' rx='10.583334' ry='9.0714283' /%3E%3C/svg%3E%0A&quot;);");
client.println("background-repeat: no-repeat;");
client.println(" }");
client.println(" </style></head><body ></body></html>");
    }

score 1 · Answer 3 · answered Jul 16 '22 at 22:10

As I understand what you want to send is basically a big string(s) stored in memory. You can quite easily store it it in external memory. You can use FRAM which is non volatile with 32K x 8 modules available for less then $5.00. They will run (Read / Write) at I2C or SPI speeds, no delays for writing or reading. There are other memory types that will also do the job but this is simple and inexpensive.

I am currently doing that with a project I am working on. It is using an Arduino Nano with about 20K of messages and several of application / program data that I store. Flash will not contain the messages and the program, not enough room. The program data is created via sensors and a rotary switch. I load the data into the FRAM sometimes in sections depending on size with my loading code. I then go to my application code which retrieves the messages from the FRAM and displays them on the LCD and terminal.

There is a pointer table in a specific location in FRAM that is pointing to each message address, size etc. This allows messages and other data to change without changing the main code. The FRAM in my case is a plug in module. This approach although a bit convoluted works nicely and you do not have to have a file structure. You can if you want. I do not know if this will work for you but it does for me.

score 0 · Answer 4 · answered Dec 05 '21 at 05:51

Gonna answer my own question again lol. I looked at spiffs, but it seemed to just be an esp thing.
anyhow, I realized that <!DOCTYPE html> probably wasn't the only doctype there is, and that svg would probably render just fine without html or css. which is true! I used <!DOCTYPE svg> and got rid of html and css, which means I didn't need to use some fancy inline css encoding, and could instead make much simpler println() statements, and still generate an accessible "webpage" written in only svg. I could probably clean up the svg too, I drew it in inkscape, which generates overly complicated svg data.

But the project works now.. in a basic sort of way anyway.

/*
An IoT Etch-A-Sketch
uses 2 rotory encoders to generate x,y data points
in a svg poly line.
*/
#include <SPI.h>
#include <NativeEthernet.h>
#include <SD.h>
File dataFile;
byte mac[] = {
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED
};
IPAddress ip(10, 1, 0, 177);
EthernetServer server(80);
//Function defs
void rotors();
void rotorA(EthernetClient client);
void rotorB(EthernetClient client);
void resetImg();  //erase datalog.txt
void Abuton();
void Bbuton();
void listenClient(EthernetClient client);
void pageWrite(EthernetClient client);
void polyLineBegin(EthernetClient client);
void startPage(EthernetClient client);
void endPage(EthernetClient client);
//int potX = A0;

//int potY = A1;
int sensorValX = 0;

int sensorValY = 0;
int oldX = 0;
int oldY = 0;
/Rotor Vars/
// Rotary Encoder Input defintions
#define CLKA 2
#define DTA 3
#define SWA 4
#define CLKB 5
#define DTB 6
#define SWB 7
int Acounter = 0;
int Bcounter = 0;
int AcurrentStateCLK;
int BcurrentStateCLK;
int AlastStateCLK;
int BlastStateCLK;
unsigned long AlastButtonPress = 0; //butons on rotory encoders
unsigned long BlastButtonPress = 0;
void setup() {
delay(10000);  //you want this delay. reason: tl;dr
  /** Set encoder pins as inputs **/
  pinMode(CLKA,INPUT);
  pinMode(DTA,INPUT);
  pinMode(SWA, INPUT_PULLUP);
  pinMode(CLKB,INPUT);
  pinMode(DTB,INPUT);
  pinMode(SWB, INPUT_PULLUP);
// Read the initial state of CLK
  AlastStateCLK = digitalRead(CLKA);
  BlastStateCLK = digitalRead(CLKB);
  /*End Rotary Encoder vars**/
Serial.begin(9600);
Ethernet.begin(mac, ip);
if (Ethernet.linkStatus() == LinkOFF) {
    Serial.println("Ethernet cable is not connected.");
  }
// start the server
  server.begin();
  delay(100);
  Serial.print("server is at ");
  Serial.println(Ethernet.localIP());
  Serial.print("Initializing SD card...");
if (!SD.begin(BUILTIN_SDCARD)) {
    Serial.println("SD Card init failed!");
    while (1); 
    }
/**SD CARD and Ethernet intialized***/

//if data file (on SD Card) Doesn't exist, create it. 
//if data file (on SD Card) does exist errase it and create a new one
  if (SD.exists("datalog.txt")) {
     Serial.println("datalog.txt exists: \n Removing it.");
     while(SD.remove("datalog.txt")!=1){
       Serial.println("Deleeting old file data");
       if (SD.remove("datalog.txt")==1){
          Serial.println ("old dataFile removed");
       }
     }

  } else {
    Serial.println("datalog.txt doesn't exist.");
         }
// open a new file and immediately close it:
  Serial.println("Creating datalog.txt...");
  dataFile = SD.open("datalog.txt", FILE_WRITE);
  dataFile.close();
// Check to see if the file exists:
  if (SD.exists("datalog.txt")) {
    Serial.println("datalog.txt exists.");
  } else {
    Serial.println("datalog.txt wasn't created.");
         }
} //end setup
void loop() {
 // listen for incoming clients
  EthernetClient client = server.available();
rotors();
listenClient(client);
}  //end loop

void listenClient(EthernetClient client){
if (client) {
    boolean currentLineIsBlank = true;
    while (client.connected()) {
       if (client.available()) {
          char c = client.read();
       // Serial.write(c);   //tells about the client connection
        if (c == '\n' && currentLineIsBlank) {
          pageWrite(client);
          break;
        }
        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        } else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }//end avail 
    } //end conect
    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
  } //end client
  }//end listener()
void pageWrite(EthernetClient client){
          startPage(client);

          polyLineBegin(client);
          endPage(client);

  }
void startPage(EthernetClient client){
     // doctype svg no need for html
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connection: close");  // the connection will be closed after completion of the response
          client.println("Refresh: 1");  // refresh the page automatically every 5 sec
          client.println();
          client.println("<!DOCTYPE svg>");
          client.println("<svg   xmlns='http://www.w3.org/2000/svg'  viewBox='0 0 227.35277 156.29323' height='6.1532764in' width='8.9508963in'><path  style='fill:#ff0000; stroke:#ff0000; stroke-width:1.32300019;stroke-miterlimit:4; stroke-dasharray:none; stroke-opacity:1' d='m 12.000819,0.66145838 c -6.2819637,0 -11.33936062,5.05739692 -11.33936062,11.33936062 V 144.29249 c 0,6.28197 5.05739692,11.33936 11.33936062,11.33936 H 215.35224 c 6.28197,0 11.33885,-5.05739 11.33885,-11.33936 V 12.000819 c 0,-6.2819637 -5.05688,-11.33936062 -11.33885,-11.33936062 z M 18.932696,12.696383 H 205.5854 c 5.02557,0 9.07128,4.04571 9.07128,9.071282 v 77.03923 c 0,5.025575 -4.04571,9.071285 -9.07128,9.071285 H 18.932696 c -5.025572,0 -9.0712819,-4.04571 -9.0712819,-9.071285 v -77.03923 c 0,-5.025572 4.0457099,-9.071282 9.0712819,-9.071282 z' /><ellipse style='fill:#ffffff;fill-opacity:1; stroke:#ff0000; stroke-width:1.32300007; stroke-miterlimit:4; stroke-dasharray:none; stroke-opacity:1' cx='21.16666'  cy='130.59085'  rx='10.583334' ry='9.0714283' /><ellipse  style='fill:#ffffff; fill-opacity:1; stroke:#ff0000; stroke-width:1.32300007; stroke-miterlimit:4; stroke-dasharray:none;stroke-opacity:1' cx='200.62982'   cy='131.57361' rx='10.583334' ry='9.0714283'/>");
          client.println("\n<polyline points='15,15");

   }
 void endPage(EthernetClient client){
  client.print("' \nfill='none' stroke='#ff0000'/></svg>");
   }
void polyLineBegin(EthernetClient client){
//read data points from SD card
    File dataFile = SD.open("datalog.txt");
  // if the file is available, read it:
  if (dataFile) {
    while (dataFile.available()) {
      //Serial.write(dataFile.read());
      client.write(dataFile.read());
    }
    dataFile.close();
  }
}
int rotorA(){ //X coords
  AcurrentStateCLK = digitalRead(CLKA);
  if (AcurrentStateCLK != AlastStateCLK  && AcurrentStateCLK == 1){
     if (digitalRead(DTA) != AcurrentStateCLK) {
      Acounter --;
      }else{
       Acounter ++;
       }
     }
// Remember last CLK state
  AlastStateCLK = AcurrentStateCLK;
Abuton(); //errase the data file if A button pressed.

//X limits
if(Acounter<=15){
  Acounter=15;
  }
if(Acounter>=215){
  Acounter=215;
  }

return Acounter;
  }//endA rotor
int rotorB(){ //Y coords
  BcurrentStateCLK = digitalRead(CLKB);
  if (BcurrentStateCLK != BlastStateCLK  && BcurrentStateCLK == 1){
    if (digitalRead(DTB) != BcurrentStateCLK) {
       Bcounter --;
       }else{
        Bcounter ++;
     }
    //Serial.println(Bcounter);
  }

  BlastStateCLK = BcurrentStateCLK;
Bbuton();
//Y limits
if(Bcounter<=15){
  Bcounter=15;
  }
if(Bcounter>=210){
  Bcounter=210;
  }

  return Bcounter;
  }//EndB rotor
void rotors(){
//read rotors and write x,y points to datalog.txt

 sensorValX=rotorA();
 sensorValY=rotorB();
String dataString = "";
if(sensorValX!=oldX || sensorValY!=oldY){
   dataString += ",";
   dataString += String(sensorValX);
   dataString += ", ";
   dataString += String(sensorValY);
Serial.println(dataString);
dataFile = SD.open("datalog.txt", FILE_WRITE);
    // if the file is available, write to it:
   if (dataFile) {
      dataFile.print(dataString);
      dataFile.close();
      //clear dataString
      dataString="";
      }else{
    // if the file isn't open, pop up an error:
    Serial.println("error opening datalog.txt");
  }
  dataFile.close();
  }else{
  ;
  }
oldX=sensorValX;
 oldY=sensorValY;
}//end rotors 
void resetImg(){
  //clear datapoints by errasing datafile.
  if (SD.exists("datalog.txt")) {
     Serial.println("Deleting data.");
     while(SD.remove("datalog.txt")!=1){
       Serial.println("Deleeting old file data");
       if (SD.remove("datalog.txt")==1){
          Serial.println ("old dataFile removed");
       }
     }

  } else {
    Serial.println("datalog.txt doesn't exist.");
         }
// open a new file and immediately close it:
  Serial.println("Creating datalog.txt...");
  dataFile = SD.open("datalog.txt", FILE_WRITE);
  dataFile.close();
// Check to see if the file exists:
  if (SD.exists("datalog.txt")) {
    Serial.println("New datalog.txt created.");
  } else {
    Serial.println("datalog.txt wasn't created.");
         }
}//end resetImg
void Abuton(){ //X buton
    int AbtnState = digitalRead(SWA);
  //If we detect LOW signal, button is pressed
if (AbtnState == LOW) {
    if (millis() - AlastButtonPress > 500) {
      Serial.println("Resting image!");
      resetImg();
    }
  AlastButtonPress = millis();
  }
  } //end Abuton
void Bbuton(){ //Y buton
int BbtnState = digitalRead(SWB);
    if (BbtnState == LOW) {
    //if 50ms have passed since last LOW pulse, it means that the
    //button has been pressed, released and pressed again
    if (millis() - BlastButtonPress > 50) {
      Serial.println("Button B pressed!");
    }

    BlastButtonPress = millis();
  }
}

ocrdu · Answer 5 · 2022-07-15T17:31:17.367

Here's an alternative way of doing things I like to use:

What I like to do, when working with Arduinos with a decent amount of memory like the SAMD21-based ones, is create a fully self-contained webpage, that is, a webpage that has all HTML, CSS and Javascript in the same file, and all bitmap and SVG images in-lined in that same file. In short, a webpage file that is completely self-contained.

I then gzip that file (which compresses it a lot), and then base64 encode the gzipped file. The base64-encoded page is then copy/pasted into the code / memory as a const char*.

When the page needs to be served, I base64-decode it, and then serve the gzipped version, which all browsers accept if you put Content-Encoding: gzip in the header.

I put an elaborate example on Github, maybe you can get some inspiration from that. The webpage in the example is 26.8 kB in size, and the base64-encoded, gzipped webpage string as put into the code comes in at a little under 7 kB.

How to make better looking webpages on an Arduino webserver?

5 Answers5