Canvas-based body chart annotation: how it was built

The instinctive first approach is to use an HTML <img> tag with a body diagram image and let physicians draw on top using CSS or SVG overlays. This works for simple marking but quickly breaks down:

• SVG overlays are hard to make feel natural for free-form drawing
• CSS-based annotation cannot capture brush pressure or natural stroke paths
• Simple pixel overlay approaches do not support erasure properly
• Storing annotation data in a structured way (for export to PDF or for future editing) requires canvas serialization

The HTML5 Canvas API is the right tool for this. Combined with Fabric.js (a canvas library that adds object management, selection, and serialization on top of the raw Canvas API), it provides what is needed: natural drawing, proper erasure, canvas serialization to JSON for storage, and export to base64 images for PDF generation.

HTML5 Canvas API provides the drawing surface. The canvas element is a raster image surface where each pixel can be manipulated directly. For drawing, the Canvas 2D rendering context provides paths, stroke operations, and compositing modes.

Fabric.js is a layer on top of the Canvas API that provides:

• Object model (paths, images, text are objects on the canvas, not just pixels)
• Selection and manipulation of drawn objects
• Canvas serialization to JSON (the entire canvas state can be saved as a JavaScript object)
• Canvas import from JSON (the saved state can be restored exactly)
• Export to base64 (for PDF generation and display)

The object model is particularly important for erasure. With raw canvas, erasing requires painting over drawn pixels with the background color. This looks wrong when the background is a complex body diagram image. With Fabric.js's composite modes, you can draw with the "destination-out" composite mode, which erases any pixels the eraser touches, revealing the layer below cleanly.

The first challenge is integrating Fabric.js with React without the two systems interfering with each other.

React manages the DOM through its virtual DOM model. It expects to be the only thing writing to DOM nodes in a React component. Fabric.js also writes directly to the canvas DOM element to handle rendering. If both try to manage the same DOM node, conflicts occur.

The solution is to keep the canvas element outside React's management scope:

1. 1Render an empty <canvas> element in JSX with a stable ref
2. 2In a useEffect with empty dependencies, initialize the Fabric canvas using the ref
3. 3Store the Fabric canvas instance in a separate ref (not React state, to avoid re-renders)
4. 4All subsequent interaction with the canvas happens through the Fabric API using the stored ref
5. 5React state is used only for tool selection, color, and brush size — not for the canvas content itself

This pattern keeps React and Fabric.js separated: React manages the non-canvas UI elements, Fabric.js manages the canvas content, and the ref bridges them.

The body chart annotation tool has four primary tools:

Pen. Free-form path drawing. Using Fabric.js's PencilBrush, the physician draws directly on the canvas with their mouse or touch input. The brush has configurable color and width. As the physician draws, the path is created as a Fabric Path object on the canvas.

Eraser. Using the destination-out composite rendering mode, the eraser removes any pixels the brush touches, revealing the underlying body diagram. The eraser uses the same brush width control as the pen. In Fabric.js, this is implemented by setting the brush color to an opaque color and the globalCompositeOperation to "destination-out" on the rendering context.

Color picker. Physicians can choose from a palette of colors or use a full color picker. Different colors serve different documentation purposes: red for pain, blue for numbness, black for injury location, green for treatment area. The color applies to the next pen stroke.

Brush size control. A slider controls the brush width for both pen and eraser tools. Finer strokes for precise marking, wider strokes for marking larger regions.

One size does not fit all documentation contexts. A small illustration in a report section and a full-page body diagram used as the primary documentation tool have different resolution requirements.

The Medico body chart supports three canvas sizes:

• Small (670 × 500 pixels). Suitable for inline documentation within a larger report, where the body chart is one element among several on a page.
• Medium (1024 × 1024 pixels). Standard documentation size, suitable for body charts that are a significant portion of the report.
• Large (1400 × 1400 pixels). High-resolution documentation for legal exhibits and formal medical reports where image clarity at print resolution matters.

The size is configured in the template element settings when the clinic designs the template. It is fixed for a given template element and cannot be changed per-patient instance (changing resolution would destroy existing annotations on that element).

In chiropractic and pain management documentation, marking a body region is often not enough. Physicians need to document which specific spinal levels are involved: which cervical vertebrae (C1-C7), which thoracic vertebrae (T1-T12), and which lumbar vertebrae (L1-L5).

A free-form body chart annotation can mark the general spinal region, but it cannot produce structured data about specific vertebral levels. The spine level selector complements the body chart by providing a visual vertebral column where each level is a selectable element.

The physician clicks on C5, T3, or L2, and those levels are highlighted. The selection is stored as a structured array of level identifiers, not as a canvas image. This means the selected levels can be:

• Displayed in the report template
• Exported as text in the PDF report ("Affected levels: C5, C6, T1, T2")
• Searched and filtered (find all patients with L4-L5 involvement)

The visual design of the spine selector shows a simplified vertebral column diagram with clickable regions for each level. Selected levels are highlighted in a distinct color. Side-by-side checkboxes for left and right allow documenting unilateral vs bilateral involvement at each level.

When a physician saves their body chart annotation, the data needs to be stored in a way that supports:

• Exact restoration of the annotation for later editing
• Export to an image for PDF generation
• Future display in the completed report

Fabric.js provides two serialization methods:

• canvas.toJSON() serializes the entire canvas state as a JavaScript object, including all paths, their positions, colors, and composite modes. This enables exact restoration.
• canvas.toDataURL() exports the current canvas state as a base64 PNG. This is used for PDF embedding and report display.

The Medico platform stores both. The JSON representation is stored for editing (when the physician needs to reopen and modify an existing annotation). The base64 PNG is stored for display and PDF export.

Storage is in AWS S3 for the images, with the JSON metadata stored in the report element record in the database.

In treatment documentation, showing the change in injury pattern over multiple visits is clinically valuable. The Medico body chart includes a comparison view that shows two body charts side by side: the initial annotation from visit one and the current annotation from the most recent visit.

This is rendered as a flexbox layout with two canvas elements side by side, each initialized with the stored Fabric JSON from its respective visit. The comparison view is read-only (no editing in comparison mode) and is available in the report summary view.

When a completed medical report is exported as a PDF, the body chart annotation appears in the appropriate position in the document layout.

The base64 PNG of the annotation is embedded directly in the PDF as an image. PDF generation (using pdfkit server-side) receives the report data including the base64 image string and embeds it at the configured size within the template section layout.

For the spine level selector, the PDF generation converts the array of selected level identifiers to a text string and embeds it as formatted text.