Typeset non-TeX files by converting them using Lua code

The following code example demonstrates how to use the file reading callback and coroutines to convert a non-TeX file into TeX code which is then typeset.

First the Lua code ("Preprocessor.lua"):

 local function readFile(fn)
 	local file = assert(io.open(fn, "r"))
 	local contents = file:read("*a")
 	file:close()
 	return contents
 end
 
 
 local function processInputFile(contents)
 	-- Preamble
 	coroutine.yield("\\pdfoutput=1 ")
 	
 	-- Process the file: Return each line
 	for line in contents:gmatch("(.-)[\n\r]") do
 		coroutine.yield(line)
 	end
 	
 	-- Postamble
 	coroutine.yield("\\bye")
 	
 	-- Done
 	coroutine.yield(nil)
 end
 
 
 callback.register("open_read_file", function(fileName)
 	local contents = readFile(fileName)
 	return {
 		reader = coroutine.wrap(function()
 			processInputFile(contents .. "\n")
 		end)
 	}
 end)

Then a small test file ("Preprocessor.txt"):

 Hello World1
 
 Hello World2
 
 Paragraph 3,
 continuing right now
 and going on over
 four lines

And finally the bash shell script to tie everything together ("Preprocessor.sh"):

 luatex -fmt=luatex --jobname=Preprocessor '\directlua0{dofile("Preprocessor.lua")}\input Preprocessor.txt'

As you can see, the file to be converted is simply read using "\input" and then converted in the Lua function "processInputFile". By using coroutines, we can pretend to convert the file in one go, when really we are supplying it line by line to LuaTeX.

It would of course be possible to check the file type/extension inside the "open_read_file" callback and perform different conversions (or none at all if a TeX file) depending on the file type, i.e. to typeset source code.