/linkedfs/usr/share/nmap/nselib/smb.lua

Large files files are truncated, but you can click here to view the full file

---Implements functionality related to Server Message Block (SMB, also known 
-- as CIFS) traffic, which is a Windows protocol.
--
-- SMB traffic is normally sent to/from ports 139 or 445 of Windows systems. Other systems
-- implement SMB as well, including Samba and a lot of embedded devices. Some of them implement
-- it properly and many of them not. Although the protocol has been documented decently 
-- well by Samba and others, many 3rd party implementations are broken or make assumptions. 
-- Even Samba's and Windows' implementations aren't completely compatiable. As a result, 
-- creating an implementation that accepts everything is a bit of a minefield. 
--
-- Where possible, this implementation, since it's intended for scanning, will attempt to 
-- accept any invalid implementations it can, and fail gracefully if it can't. This has
-- been tested against a great number of weird implementations, and it now works against 
-- all of them. 
--
-- The intention of this library is to eventually handle all aspects of the SMB protocol.
-- That being said, I'm only implementing the pieces that I (Ron Bowes) need. If you 
-- require something more, let me know and I'll put it on my todo list. 
--
-- A programmer using this library should already have some knowledge of the SMB protocol, 
-- although a lot isn't necessary. You can pick up a lot by looking at the code. The basic
-- login/logoff is this:
--
--<code>
-- [connect]
-- C->S SMB_COM_NEGOTIATE
-- S->C SMB_COM_NEGOTIATE
-- C->S SMB_COM_SESSION_SETUP_ANDX
-- S->C SMB_COM_SESSION_SETUP_ANDX
-- C->S SMB_COM_TREE_CONNECT_ANDX
-- S->C SMB_COM_TREE_CONNECT_ANDX
-- ...
-- C->S SMB_COM_TREE_DISCONNECT
-- S->C SMB_COM_TREE_DISCONNECT
-- C->S SMB_COM_LOGOFF_ANDX
-- S->C SMB_COM_LOGOFF_ANDX
-- [disconnect]
--</code>
--
-- In terms of functions here, the protocol is:
--
--<code>
-- status, smbstate = smb.start(host)
-- status, err      = smb.negotiate_protocol(smbstate, {})
-- status, err      = smb.start_session(smbstate, {})
-- status, err      = smb.tree_connect(smbstate, path, {})
-- ...
-- status, err      = smb.tree_disconnect(smbstate)
-- status, err      = smb.logoff(smbstate)
-- status, err      = smb.stop(smbstate)
--</code>
--
-- The <code>stop</code> function will automatically call tree_disconnect and logoff, 
-- cleaning up the session, if it hasn't been done already.
-- 
-- To initially begin the connection, there are two options:
--
-- 1) Attempt to start a raw session over 445, if it's open.
--
-- 2) Attempt to start a NetBIOS session over 139. Although the 
--    protocol's the same, it requires a <code>session request</code> packet. 
--    That packet requires the computer's name, which is requested
--    using a NBSTAT probe over UDP port 137. 
--
-- Once it's connected, a <code>SMB_COM_NEGOTIATE</code> packet is sent, requesting the protocol 
-- "NT LM 0.12", which is the most commonly supported one. Among other things, the server's 
-- response contains the host's security level, the system time, and the computer/domain name. 
-- Some systems will refuse to use that protocol and return "-1" or "1" instead of 0. If that's 
-- detected, we kill the connection (because the protocol following won't work). 
--
-- If that's successful, <code>SMB_COM_SESSION_SETUP_ANDX</code> is sent. It is essentially the logon
-- packet, where the username, domain, and password are sent to the server for verification. 
-- The username and password are generally picked up from the program parameters, which are
-- set when running a script, or from the registry where it can be set by other scripts (for 
-- example, <code>smb-brute.nse</code>). However, they can also be passed as parameters to the 
-- function, which will override any other username/password set. 
--
-- If a username and password are set, they are used for the first login attempt. If a login fails,
-- or they weren't set, a connection as the 'GUEST' account with a blank password is attempted. If
-- that fails, then a NULL session is established, which should always work. The username/password
-- will give the highest access level, GUEST will give lower access, and NULL will give the lowest
-- (often, NULL will give no access). 
--
-- The actual login protocol used by <code>SMB_COM_SESSION_SETUP_ANDX</code> is explained in detail
-- in <code>smbauth.lua</code>. 
--
-- Thanks go to Christopher R. Hertel and his book Implementing CIFS, which 
-- taught me everything I know about Microsoft's protocols. Additionally, I used Samba's
-- list of error codes for my constants. Although I don't believe they would be covered
-- by GPL, since they're public now anyways, but I'm not a lawyer and, if somebody feels
-- differently, let me know and we can sort this out. 
--
-- Scripts that use this module can use the script arguments listed below
-- example of using these script arguments:
-- <code>
-- nmap --script=smb-<script>.nse --script-args=smbuser=ron,smbpass=iagotest2k3,smbbasic=1,smbsign=force <host>
-- </code>
-- 
--@args  smbbasic    Forces the authentication to use basic security, as opposed to "extended security". 
--                   Against most modern systems, extended security should work, but there may be cases
--                   where you want to force basic. There's a chance that you'll get better results for 
--                   enumerating users if you turn on basic authentication. 
--@args smbsign      Controls whether or not server signatures are checked in SMB packets. By default, on Windows,
--                   server signatures aren't enabled or required. By default, this library will always sign 
--                   packets if it knows how, and will check signatures if the server says to. Possible values are:
-- * <code>force</code>:      Always check server signatures, even if server says it doesn't support them (will 
--                           probably fail, but is technically more secure). 
-- * <code>negotiate</code>: [default] Use signatures if server supports them. 
-- * <code>ignore</code>:    Never check server signatures. Not recommended. 
-- * <code>disable</code>:   Don't send signatures, at all, and don't check the server's. not recommended. 
--                   More information on signatures can be found in <code>smbauth.lua</code>.
--@args smbport      Override the default port choice. If <code>smbport</code> is open, it's used. It's assumed
--                   to be the same protocol as port 445, not port 139. Since it probably isn't possible to change
--                   Windows' ports normally, this is mostly useful if you're bouncing through a relay or something. 
--@args randomseed   Set to a value to change the filenames/service names that are randomly generated. 
--                   
--@author Ron Bowes <ron@skullsecurity.net>
--@copyright Same as Nmap--See http://nmap.org/book/man-legal.html
-----------------------------------------------------------------------
module(... or "smb", package.seeall)

require 'bit'
require 'bin'
require 'netbios'
require 'smbauth'
require 'stdnse'

-- These arrays are filled in with constants at the bottom of this file
command_codes = {}
command_names = {}
status_codes = {}
status_names = {}

local mutexes = setmetatable({}, {__mode = "k"});
--local debug_mutex = nmap.mutex("SMB-DEBUG")

local TIMEOUT = 10000

---Wrapper around <code>smbauth.add_account</code>.
function add_account(host, username, domain, password, password_hash, hash_type, is_admin)
	smbauth.add_account(host, username, domain, password, password_hash, hash_type, is_admin)
end

---Wrapper around <code>smbauth.get_account</code>. 
function get_account(host)
	return smbauth.get_account(host)
end
---Create an 'overrides' table
function get_overrides(username, domain, password, password_hash, hash_type, overrides)
	if(not(overrides)) then
		return {username=username, domain=domain, password=password, password_hash=password_hash, hash_type=hash_type}
	else
		overrides['username'] = username
		overrides['domain'] = domain
		overrides['password'] = password
		overrides['password_hash'] = password_hash
		overrides['hash_type'] = hash_type
	end
end

---Get an 'overrides' table for the anonymous user
--
--@param overrides [optional] A base table of overrides. The appropriate fields will be added. 
function get_overrides_anonymous(overrides)
	if(not(overrides)) then
		return {username='', domain='', password='', password_hash=nil, hash_type='none'}
	else
		overrides['username'] = ''
		overrides['domain'] = ''
		overrides['password'] = ''
		overrides['password_hash'] = ''
		overrides['hash_type'] = 'none'
	end
end


---Returns the mutex that should be used by the current connection. This mutex attempts
-- to use the name, first, then falls back to the IP if no name was returned. 
--
--@param smbstate The SMB object associated with the connection
--@return A mutex
local function get_mutex(smbstate)
	local mutex_name = "SMB-"
	local mutex

--	if(nmap.debugging() > 0) then
--		return debug_mutex
--	end

	-- Decide whether to use the name or the ip address as the unique identifier
	if(smbstate['name'] ~= nil) then
		mutex_name = mutex_name .. smbstate['name']
	else
		mutex_name = mutex_name .. smbstate['ip']
	end

	if(mutexes[smbstate] == nil) then
		mutex = nmap.mutex(mutex_name)
		mutexes[smbstate] = mutex
	else
		mutex = mutexes[smbstate]
	end

	stdnse.print_debug(3, "SMB: Using mutex named '%s'", mutex_name)

	return mutex
end

---Locks the mutex being used by this host. Doesn't return until it successfully 
-- obtains a lock. 
--
--@param smbstate The SMB object associated with the connection
--@param func     A name to associate with this call (used purely for debugging
--                and logging)
local function lock_mutex(smbstate, func)
	local mutex

	stdnse.print_debug(3, "SMB: Attempting to lock mutex [%s]", func)
	mutex = get_mutex(smbstate)
	mutex "lock"
	stdnse.print_debug(3, "SMB: Mutex lock obtained [%s]", func)
end

---Unlocks the mutex being used by this host.
--
--@param smbstate The SMB object associated with the connection
--@param func     A name to associate with this call (used purely for debugging
--                and logging)
local function unlock_mutex(smbstate, func)
	local mutex

	stdnse.print_debug(3, "SMB: Attempting to release mutex [%s]", func)
	mutex = get_mutex(smbstate)
	mutex "done"
	stdnse.print_debug(3, "SMB: Mutex released [%s]", func)
end

---Convert a status number from the SMB header into a status name, returning an error message (not nil) if 
-- it wasn't found. 
--
--@param status The numerical status. 
--@return A string representing the error. Never nil. 
function get_status_name(status)

	if(status_names[status] == nil) then
		-- If the name wasn't found in the array, do a linear search on it
		for i, v in pairs(status_names) do
			if(v == status) then
				return i
			end
		end

		return string.format("NT_STATUS_UNKNOWN (0x%08x)", status)
	else
		return status_names[status]
	end
end


--- Determines whether or not SMB checks are possible on this host, and, if they are, 
--  which port is best to use. This is how it decides:
--
-- * If port tcp/445 is open, use it for a raw connection
-- * Otherwise, if ports tcp/139 and udp/137 are open, do a NetBIOS connection. Since
--   UDP scanning isn't default, we're also ok with udp/137 in an unknown state. 
--
--@param host The host object. 
--@return The port number to use, or nil if we don't have an SMB port
function get_port(host)
	local port_u137 = nmap.get_port_state(host, {number=137, protocol="udp"})
	local port_t139 = nmap.get_port_state(host, {number=139, protocol="tcp"})
	local port_t445 = nmap.get_port_state(host, {number=445, protocol="tcp"})
	local custom_port = nil

	if(nmap.registry.args.smbport ~= nil) then
		custom_port = nmap.get_port_state(host, {number=tonumber(nmap.registry.args.smbport), protocol="tcp"})
	end

	-- Try a user-defined port first
	if(custom_port ~= nil and custom_port.state == "open") then
		return custom_port.number
	end

	if(port_t445 ~= nil and port_t445.state == "open") then
		 -- tcp/445 is open, we're good
		 return 445
	end

	if(port_t139 ~= nil and port_t139.state == "open") then
		 -- tcp/139 is open, check uf udp/137 is open or unknown
		 if(port_u137 == nil or port_u137.state == "open" or port_u137.state == "open|filtered") then
			  return 139
		 end
	end

	return nil
end

---Turn off extended security negotiations for this connection. There are a few reasons you might want to
-- do that, the main ones being that extended security is going to be marginally slower and it's not going
-- to give the same level of information in some cases (namely, it doesn't present the server's name). 
--@param smb The SMB state table. 
function disable_extended(smb)
	smb['extended_security'] = false
end

--- Begins a SMB session, automatically determining the best way to connect. Also starts a mutex
--  with mutex_id. This prevents multiple threads from making queries at the same time (which breaks
--  SMB). 
--
-- @param host The host object
-- @param overrides [optional] Overrides for various fields
-- @return (status, smb) if the status is true, result is the newly crated smb object; 
--         otherwise, socket is the error message. 
function start(host)
	local port = get_port(host)
	local status, result
	local state = {}

	state['uid']      = 0
	state['tid']      = 0
	state['host']     = host
	state['ip']       = host.ip
	state['sequence'] = -1

	-- Check whether or not the user requested basic authentication
	if(nmap.registry.args.smbbasic == nil) then
		state['extended_security'] = true
	else
		state['extended_security'] = false
	end

	-- Store the name of the server
	status, result = netbios.get_server_name(host.ip)
	if(status == true) then
		state['name'] = result
	end

	stdnse.print_debug(2, "SMB: Starting SMB session for %s (%s)", host.name, host.ip)

	if(port == nil) then
		return false, "SMB: Couldn't find a valid port to check"
	end

	-- Initialize the accounts for logging on (note: this has to be outside the mutex, or things break)
	smbauth.init_account(host)

	lock_mutex(state, "start(1)")

	if(port ~= 139) then
		status, state['socket'] = start_raw(host, port)
		state['port'] = port

		if(status == false) then
			unlock_mutex(state, "start(1)")
			return false, state['socket']
		end
		return true, state

	else
		status, state['socket'] = start_netbios(host, port)
		state['port'] = port
		if(status == false) then
			unlock_mutex(state, "start(2)")
			return false, state['socket']
		end
		return true, state

	end

	unlock_mutex(state, "start(3)")

	return false, "SMB: Couldn't find a valid port to check"
end

---Initiates a SMB connection over whichever port it can, then optionally sends the common
-- initialization packets. Note that each packet depends on the previous one, so if you want
-- to go all the way up to create_file, you have to set all parameters. 
--
-- If anything fails, we back out of the connection and return an error, so the calling functino
-- doesn't have to call smb.stop(). 
--
--@param host               The host object. 
--@param negotiate_protocol [optional] If 'true', send the protocol negotiation. Default: false. 
--@param start_session      [optional] If 'true', start the session. Default: false. 
--@param tree_connect       [optional] The tree to connect to, if given (eg. "IPC$" or "C$"). If not given, 
--                          packet isn't sent. 
--@param create_file        [optional] The path and name of the file (or pipe) that's created, if given. If 
--                          not given, packet isn't sent. 
--@param overrides          [optional] A table of overrides (for, for example, username, password, etc.) to pass 
--                          to all functions. 
--@param disable_extended   [optional] If set to true, disables extended security negotiations. 
function start_ex(host, negotiate_protocol, start_session, tree_connect, create_file, disable_extended, overrides)
	local smbstate
	local status, err

	-- Make sure we have overrides
	overrides = overrides or {}

	-- Begin the SMB session
	status, smbstate = smb.start(host)
	if(status == false) then
		return false, smbstate
	end

	-- Disable extended security if it was requested
	if(disable_extended == true) then
		smb.disable_extended(smbstate)
	end

	if(negotiate_protocol == true) then
		-- Negotiate the protocol
		status, err = smb.negotiate_protocol(smbstate, overrides)
		if(status == false) then
			smb.stop(smbstate)
			return false, err
		end

		if(start_session == true) then	
			-- Start up a session
			status, err = smb.start_session(smbstate, overrides)
			if(status == false) then
				smb.stop(smbstate)
				return false, err
			end

			if(tree_connect ~= nil) then
				-- Connect to share
				status, err = smb.tree_connect(smbstate, tree_connect, overrides)
				if(status == false) then
					smb.stop(smbstate)
					return false, err
				end

				if(create_file ~= nil) then
					-- Try to connect to requested pipe
					status, err = smb.create_file(smbstate, create_file, overrides)
					if(status == false) then
						smb.stop(smbstate)
						return false, err
					end
				end
			end
		end
	end

	-- Return everything
	return true, smbstate
end

--- Kills the SMB connection, closes the socket, and releases the mutex. Because of the mutex 
--  being released, a script HAS to call <code>stop</code> before it exits, no matter why it's exiting! 
--
--  In addition to killing the connection, this function will log off the user and disconnect
--  the connected tree, if possible.
--
--@param smb    The SMB object associated with the connection
--@return (status, result) If status is false, result is an error message. Otherwise, result
--        is undefined. 
function stop(smb) 

	if(smb['tid'] ~= 0) then
		tree_disconnect(smb)
	end

	if(smb['uid'] ~= 0) then
		logoff(smb)
	end

	unlock_mutex(smb, "stop()")

	stdnse.print_debug(2, "SMB: Closing socket")
	if(smb['socket'] ~= nil) then
		local status, err = smb['socket']:close()

		if(status == false) then
			return false, "SMB: Failed to close socket: " .. err
		end
	end

	return true
end

--- Begins a raw SMB session, likely over port 445. Since nothing extra is required, this
--  function simply makes a connection and returns the socket. 
-- 
--@param host The host object to check. 
--@param port The port to use (most likely 445).
--@return (status, socket) if status is true, result is the newly created socket. 
--        Otherwise, socket is the error message. 
function start_raw(host, port)
	local status, err
	local socket = nmap.new_socket()

	socket:set_timeout(TIMEOUT)
	status, err = socket:connect(host.ip, port, "tcp")

	if(status == false) then
		return false, "SMB: Failed to connect to host: " .. err
	end

	return true, socket
end

--- This function will take a string like "a.b.c.d" and return "a", "a.b", "a.b.c", and "a.b.c.d". 
--  This is used for discovering NetBIOS names. If a NetBIOS name is unknown, the substrings of the 
--  DNS name can be used in this way. 
--
--@param name The name to take apart
--@return An array of the sub names
local function get_subnames(name)
	local i = -1
	local list = {}

	repeat
		local subname = name

		i = string.find(name, "[.]", i + 1)
		if(i ~= nil) then
			subname = string.sub(name, 1, i - 1)
		end

		list[#list + 1] = string.upper(subname)

	until i == nil

	return list
end

--- Begins a SMB session over NetBIOS. This requires a NetBIOS Session Start message to 
--  be sent first, which in turn requires the NetBIOS name. The name can be provided as
--  a parameter, or it can be automatically determined. 
--
-- Automatically determining the name is interesting, to say the least. Here are the names
-- it tries, and the order it tries them in:
-- * The name the user provided, if present
-- * The name pulled from NetBIOS (udp/137), if possible
-- * The generic name "*SMBSERVER"
-- * Each subset of the domain name (for example, scanme.insecure.org would attempt "scanme",
--    "scanme.insecure", and "scanme.insecure.org")
--
-- This whole sequence is a little hackish, but it's the standard way of doing it. 
--
--@param host The host object to check. 
--@param port The port to use (most likely 139).
--@param name [optional] The NetBIOS name of the host. Will attempt to automatically determine
--            if it isn't given. 
--@return (status, socket) if status is true, result is the port
--        Otherwise, socket is the error message. 
function start_netbios(host, port, name)
	local i
	local status, err
	local pos, result, flags, length
	local socket = nmap.new_socket()

	-- First, populate the name array with all possible names, in order of significance
	local names = {}

	-- Use the name parameter
	if(name ~= nil) then
		names[#names + 1] = name
	end

	-- Get the name of the server from NetBIOS
	status, name = netbios.get_server_name(host.ip)
	if(status == true) then
		names[#names + 1] = name
	end

	-- "*SMBSERVER" is a special name that any server should respond to
	names[#names + 1] = "*SMBSERVER"

	-- If all else fails, use each substring of the DNS name (this is a HUGE hack, but is actually
	-- a recommended way of doing this!)
	if(host.name ~= nil and host.name ~= "") then
		local new_names = get_subnames(host.name)
		for i = 1, #new_names, 1 do
			names[#names + 1] = new_names[i]
		end
	end

	-- This loop will try all the NetBIOS names we've collected, hoping one of them will work. Yes,
	-- this is a hackish way, but it's actually the recommended way. 
	i = 1
	repeat

		-- Use the current name
		name = names[i]

		-- Some debug information
		stdnse.print_debug(1, "SMB: Trying to start NetBIOS session with name = '%s'", name)
		-- Request a NetBIOS session
		local session_request = bin.pack(">CCSzz", 
					0x81,                        -- session request
					0x00,                        -- flags
					0x44,                        -- length
					netbios.name_encode(name),   -- server name
					netbios.name_encode("NMAP")  -- client name
				);

		stdnse.print_debug(3, "SMB: Connecting to %s", host.ip)
		socket:set_timeout(TIMEOUT)
		status, err = socket:connect(host.ip, port, "tcp")
		if(status == false) then
			socket:close()
			return false, "SMB: Failed to connect: " .. err
		end

		-- Send the session request
		stdnse.print_debug(3, "SMB: Sending NetBIOS session request with name %s", name)
		status, err = socket:send(session_request)
		if(status == false) then
			socket:close()
			return false, "SMB: Failed to send: " .. err
		end
		socket:set_timeout(TIMEOUT)
	
		-- Receive the session response
		stdnse.print_debug(3, "SMB: Receiving NetBIOS session response")
		status, result = socket:receive_bytes(4);
		if(status == false) then
			socket:close()
			return false, "SMB: Failed to close socket: " .. result
		end
		pos, result, flags, length = bin.unpack(">CCS", result)
		if(length == nil) then
			return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [1]"
		end
	
		-- Check for a position session response (0x82)
		if result == 0x82 then
			stdnse.print_debug(3, "SMB: Successfully established NetBIOS session with server name %s", name)
			return true, socket
		end

		-- If the session failed, close the socket and try the next name
		stdnse.print_debug(1, "SMB: Session request failed, trying next name")
		socket:close()
	
		-- Try the next name
		i = i + 1

	until i > #names

	-- We reached the end of our names list
	stdnse.print_debug(1, "SMB: None of the NetBIOS names worked!")
	return false, "SMB: Couldn't find a NetBIOS name that works for the server. Sorry!"
end

--- Creates a string containing a SMB packet header. The header looks like this:
-- 
--<code>
-- --------------------------------------------------------------------------------------------------
-- | 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9  8  7  6  5  4  3  2  1  0 |
-- --------------------------------------------------------------------------------------------------
-- |         0xFF           |          'S'          |        'M'            |         'B'           |
-- --------------------------------------------------------------------------------------------------
-- |        Command         |                             Status...                                 |
-- --------------------------------------------------------------------------------------------------
-- |    ...Status           |        Flags          |                    Flags2                     |
-- --------------------------------------------------------------------------------------------------
-- |                    PID_high                    |                  Signature.....               |
-- --------------------------------------------------------------------------------------------------
-- |                                        ....Signature....                                       |
-- --------------------------------------------------------------------------------------------------
-- |              ....Signature                     |                    Unused                     |
-- --------------------------------------------------------------------------------------------------
-- |                      TID                       |                     PID                       |
-- --------------------------------------------------------------------------------------------------
-- |                      UID                       |                     MID                       |
-- ------------------------------------------------------------------------------------------------- 
--</code>
--
-- All fields are, incidentally, encoded in little endian byte order.
--
-- For the purposes here, the program doesn't care about most of the fields so they're given default 
-- values. The "command" field is the only one we ever have to set manually, in my experience. The TID
-- and UID need to be set, but those are stored in the smb state and don't require user intervention. 
--
--@param smb     The smb state table. 
--@param command The command to use.
--@return A binary string containing the packed packet header. 
local function smb_encode_header(smb, command)

	-- Used for the header
	local sig = string.char(0xFF) .. "SMB"

	-- Pretty much every flags is deprecated. We set these two because they're required to be on. 
	local flags  = bit.bor(0x10, 0x08) -- SMB_FLAGS_CANONICAL_PATHNAMES | SMB_FLAGS_CASELESS_PATHNAMES
	-- These flags are less deprecated. We negotiate 32-bit status codes and long names. We also don't include Unicode, which tells
	-- the server that we deal in ASCII. 
	local flags2 = bit.bor(0x4000, 0x2000, 0x0040, 0x0001) -- SMB_FLAGS2_32BIT_STATUS | SMB_FLAGS2_EXECUTE_ONLY_READS | SMB_FLAGS2_IS_LONG_NAME | SMB_FLAGS2_KNOWS_LONG_NAMES

	-- Unless the user's disabled the security signature, add it
	if(nmap.registry.args.smbsign ~= "disable") then
		flags2 = bit.bor(flags2, 0x0004) -- SMB_FLAGS2_SECURITY_SIGNATURE
	end
		

	if(smb['extended_security'] == true) then
		flags2 = bit.bor(flags2, 0x0800) -- SMB_EXTENDED_SECURITY
	end

	-- TreeID should never ever be 'nil', but it seems to happen once in awhile so print an error
	if(smb['tid'] == nil) then
		return false, string.format("SMB: ERROR: TreeID value was set to nil on host %s", smb['ip'])
	end

	local header = bin.pack("<CCCCCICSSLSSSSS",
				sig:byte(1),  -- Header
				sig:byte(2),  -- Header
				sig:byte(3),  -- Header
				sig:byte(4),  -- Header
				command,      -- Command
				0,            -- status
				flags,        -- flags
				flags2,       -- flags2
				0,            -- extra (pid_high)
				0,            -- extra (signature)
				0,            -- extra (unused)
				smb['tid'],   -- tid
				12345,        -- pid
				smb['uid'],   -- uid
				0             -- mid
			)

	return header
end

--- Converts a string containing the parameters section into the encoded parameters string. 
-- The encoding is simple:
-- * (1 byte)   The number of 2-byte values in the parameters section
-- * (variable) The parameter section
-- This is automatically done by <code>smb_send</code>. 
-- 
-- @param parameters The parameters section. 
-- @return The encoded parameters. 
local function smb_encode_parameters(parameters)
	return bin.pack("<CA", string.len(parameters) / 2, parameters)
end

--- Converts a string containing the data section into the encoded data string. 
-- The encoding is simple:
-- * (2 bytes)  The number of bytes in the data section
-- * (variable) The data section
-- This is automatically done by <code>smb_send</code>. 
--
-- @param data The data section. 
-- @return The encoded data.
local function smb_encode_data(data)
	return bin.pack("<SA", string.len(data), data)
end

---Sign the message, if possible. This is done by replacing the signature with the sequence
-- number, creating a hash, then putting that hash in the signature location. 
--@param smb  The smb state object.
--@param body The body of the packet that's being signed.
--@return The body of the packet, with the signature in place.
local function message_sign(smb, body)
	smb['sequence'] = smb['sequence'] + 1

	if(smb['mac_key'] == nil) then
		stdnse.print_debug(3, "SMB: Not signing message (missing mac_key)")
		return body
	elseif(nmap.registry.args.smbsign == "disable") then
		stdnse.print_debug(3, "SMB: Not signing message (disabled by user)")

		return body
	end

	-- Convert the sequence number to a string
	local sequence = bin.pack("<L", smb['sequence'])
	-- Create a new string, with the sequence number in place
	local new_packet = string.sub(body, 1, 14) .. sequence .. string.sub(body, 23)
	-- Calculate the signature
	local signature = smbauth.calculate_signature(smb['mac_key'], new_packet)

	return string.sub(body, 1, 14) .. signature .. string.sub(body, 23)
end

---Check the signature of the message. This is the opposite of <code>message_sign</code>, 
-- and works the same way (replaces the signature with the sequence number, calculates
-- hash, checks)
--@param smb  The smb state object.
--@param body The body of the packet that's being checked. 
--@return A true/false value -- true if the packet was signed properly, false if it wasn't. 
local function message_check_signature(smb, body)
	smb['sequence'] = smb['sequence'] + 1

	if(smb['mac_key'] == nil) then
		stdnse.print_debug(3, "SMB: Not signing message (missing mac_key)")
		return true
	elseif(nmap.registry.args.smbsign ~= "force" and bit.band(smb['security_mode'], 0x0A) ~= 0) then
		stdnse.print_debug(3, "SMB: Not signing message (server doesn't support it -- default)")
		return true
	elseif(nmap.registry.args.smbsign == "disable" or nmap.registry.args.smbsign == "ignore") then
		stdnse.print_debug(3, "SMB: Not signing message (disabled by user)")
		return true
	end

	-- Pull out the signature that they used
	local signature  = string.sub(body, 15, 22)

	-- Turn the sequence into a string
	local sequence   = bin.pack("<L", smb['sequence'])
	-- Create a new string, with the sequence number in place
	local new_packet = string.sub(body, 1, 14) .. sequence .. string.sub(body, 23)

	-- Calculate the proper signature
	local real_signature = smbauth.calculate_signature(smb['mac_key'], new_packet)

	-- Validate the signature
	return signature == real_signature
end

--- Prepends the NetBIOS header to the packet, which is essentially the length, encoded
--  in 4 bytes of big endian, and sends it out. The length field is actually 17 or 24 bits 
--  wide, depending on whether or not we're using raw, but that shouldn't matter. 
--
--@param smb        The SMB object associated with the connection
--@param header     The header, encoded with <code>smb_get_header</code>.
--@param parameters The parameters.
--@param data       The data.
--@return (result, err) If result is false, err is the error message. Otherwise, err is
--        undefined
function smb_send(smb, header, parameters, data)
    local encoded_parameters = smb_encode_parameters(parameters)
    local encoded_data       = smb_encode_data(data)
	local body               = header .. encoded_parameters .. encoded_data
	local attempts           = 5
	local status, err

	-- Calculate the message signature
	body = message_sign(smb, body)

    local out = bin.pack(">I<A", string.len(body), body)


	repeat
		attempts = attempts - 1
		stdnse.print_debug(3, "SMB: Sending SMB packet (len: %d, attempts remaining: %d)", string.len(out), attempts)
		status, err = smb['socket']:send(out)
	until(status or (attempts == 0))

	if(attempts == 0) then
		stdnse.print_debug(1, "SMB: Sending packet failed after 5 tries! Giving up.")
	end

	return status, err
end

--- Reads the next packet from the socket, and parses it into the header, parameters, 
--  and data.
--
--@param smb The SMB object associated with the connection
--@param read_data [optional] This function will read the data section if and only if
--       this value is true. This is a workaround for a bug in the tree connect packet,
--       where the length is set incorrectly. Default: true. 
--@return (status, header, parameters, data) If status is true, the header, 
--        parameters, and data are all the raw arrays (with the lengths already
--        removed). If status is false, header contains an error message and parameters/
--        data are undefined. 
function smb_read(smb, read_data)
	local status, result
	local pos, netbios_length, length, header, parameter_length, parameters, data_length, data
	local attempts = 5

	stdnse.print_debug(3, "SMB: Receiving SMB packet")

	-- Receive the response -- we make sure to receive at least 4 bytes, the length of the NetBIOS length
	smb['socket']:set_timeout(TIMEOUT)

	repeat
		attempts = attempts - 1
		status, result = smb['socket']:receive_bytes(4);
	until(status or (attempts == 0))

	-- Make sure the connection is still alive
	if(status ~= true) then
		return false, "SMB: Failed to receive bytes after 5 attempts: " .. result
	end

	-- The length of the packet is 4 bytes of big endian (for our purposes).
	-- The NetBIOS header is 24 bits, big endian
	pos, netbios_length   = bin.unpack(">I", result)
	if(netbios_length == nil) then
		return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [2]"
	end
	-- Make the length 24 bits
	netbios_length = bit.band(netbios_length, 0x00FFFFFF)

	-- The total length is the netbios_length, plus 4 (for the length itself)
	length = netbios_length + 4

	-- If we haven't received enough bytes, try and get the rest (fragmentation!)
	if(#result < length) then
		local new_result
		local attempts = 5
		stdnse.print_debug(1, "SMB: Received a fragmented packet, attempting to receive the rest of it (got %d bytes, need %d)", #result, length)

		repeat
			attempts = attempts - 1
			status, new_result = smb['socket']:receive_bytes(netbios_length - #result)
		until(status or (attempts == 0))

		-- Make sure the connection is still alive
		if(status ~= true) then
			return false, "SMB: Failed to receive bytes after 5 attempts: " .. result
		end

		-- Append the new data to the old stuff
		result = result .. new_result
		stdnse.print_debug(1, "SMB: Finished receiving fragmented packet (got %d bytes, needed %d)", #result, length)
	end

	if(#result ~= length) then
		stdnse.print_debug(1, "SMB: ERROR: Received wrong number of bytes, there will likely be issues (recieved %d, expected %d)", #result, length)
		return false, string.format("SMB: ERROR: Didn't receive the expected number of bytes; recieved %d, expected %d. This will almost certainly cause some errors.", #result, length)
	end

	-- Check the message signature (ignoring the first four bytes, which are the netbios header)
	local good_signature = message_check_signature(smb, string.sub(result, 5))
	if(good_signature == false) then
		return false, "SMB: ERROR: Server returned invalid signature"
	end

	-- The header is 32 bytes.
	pos, header   = bin.unpack("<A32", result, pos)
	if(header == nil) then
		return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [3]"
	end

	-- The parameters length is a 1-byte value.
	pos, parameter_length = bin.unpack("<C",     result, pos)
	if(parameter_length == nil) then
		return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [4]"
	end

	-- Double the length parameter, since parameters are two-byte values. 
	pos, parameters       = bin.unpack(string.format("<A%d", parameter_length*2), result, pos)
	if(parameters == nil) then
		return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [5]"
	end

	-- The data length is a 2-byte value. 
	pos, data_length      = bin.unpack("<S",     result, pos)
	if(data_length == nil) then
		return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [6]"
	end

	-- Read that many bytes of data.
	if(read_data == nil or read_data == true) then
		pos, data             = bin.unpack(string.format("<A%d", data_length),        result, pos)
		if(data == nil) then
			return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [7]"
		end
	else
		data = nil
	end

	stdnse.print_debug(3, "SMB: Received %d bytes", string.len(result))
	return true, header, parameters, data
end

--- Sends out <code>SMB_COM_NEGOTIATE</code>, which is typically the first SMB packet sent out. 
-- Sends the following:
-- * List of known protocols
--
-- Receives:
-- * The prefered dialect
-- * The security mode
-- * Max number of multiplexed connectiosn, virtual circuits, and buffer sizes
-- * The server's system time and timezone
-- * The "encryption key" (aka, the server challenge)
-- * The capabilities
-- * The server and domain names
--
--@param smb       The SMB object associated with the connection
--@param overrides [optional] Overrides for various fields
--@return (status, result) If status is false, result is an error message. Otherwise, result is
--        nil and the following elements are added to <code>smb</code>:
--      * 'security_mode'    Whether or not to use cleartext passwords, message signatures, etc.
--      * 'max_mpx'          Maximum number of multiplexed connections
--      * 'max_vc'           Maximum number of virtual circuits
--      * 'max_buffer'       Maximum buffer size
--      * 'max_raw_buffer'   Maximum buffer size for raw connections (considered obsolete)
--      * 'session_key'      A value that's basically just echoed back
--      * 'capabilities'     The server's capabilities
--      * 'time'             The server's time (in UNIX-style seconds since 1970)
--      * 'date'             The server's date in a user-readable format
--      * 'timezone'         The server's timezone, in hours from UTC
--      * 'timezone_str'     The server's timezone, as a string
--      * 'server_challenge' A random string used for challenge/response
--      * 'domain'           The server's primary domain
--      * 'server'           The server's name
function negotiate_protocol(smb, overrides)
	local header, parameters, data
	local pos
	local header1, header2, header3, ehader4, command, status, flags, flags2, pid_high, signature, unused, pid, mid

	header     = smb_encode_header(smb, command_codes['SMB_COM_NEGOTIATE'])

	-- Make sure we have overrides
	overrides = overrides or {}

	-- Parameters are blank
	parameters = ""

	-- Data is a list of strings, terminated by a blank one. 
	if(overrides['dialects'] == nil) then
		data       = bin.pack("<CzCz", 2, (overrides['dialect'] or "NT LM 0.12"), 2, "")
	else
		data = ""
		for _, v in ipairs(overrides['dialects']) do
			data = data .. bin.pack("<Cz", 2, v)
		end
		data = data .. bin.pack("Cz", 2, "")
	end

	-- Send the negotiate request
	stdnse.print_debug(2, "SMB: Sending SMB_COM_NEGOTIATE")
	local result, err = smb_send(smb, header, parameters, data)
	if(status == false) then
		return false, err
	end

	-- Read the result
	status, header, parameters, data = smb_read(smb)
	if(status ~= true) then
		return false, header
	end

	-- Parse out the header
    local uid, tid, header4
	pos, header1, header2, header3, header4, command, status, flags, flags2, pid_high, signature, unused, tid, pid, uid, mid = bin.unpack("<CCCCCICSSlSSSSS", header)

	-- Get the protocol version
	local protocol_version = string.char(header1, header2, header3, header4)
	if(protocol_version == (string.char(0xFE) .. "SMB")) then
		return false, "SMB: Server returned a SMBv2 packet, don't know how to handle"
	end

	-- Check if we fell off the packet (if that happened, the last parameter will be nil)
	if(mid == nil) then
		return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [8]"
	end

	-- Since this is the first response seen, check any necessary flags here
	if(bit.band(flags2, 0x0800) ~= 0x0800) then
		smb['extended_security'] = false
	end

	-- Parse the parameter section
	pos, smb['dialect'] = bin.unpack("<S", parameters)
	if(smb['dialect'] == nil) then
		return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [9]"
	end

	-- Check if we ran off the packet
	if(smb['dialect'] == nil) then
		return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [10]"
	end
	-- Check if the server didn't like our requested protocol
	if(smb['dialect'] ~= 0) then
		return false, string.format("Server negotiated an unknown protocol (#%d) -- aborting", smb['dialect'])
	end

	pos, smb['security_mode'], smb['max_mpx'], smb['max_vc'], smb['max_buffer'], smb['max_raw_buffer'], smb['session_key'], smb['capabilities'], smb['time'], smb['timezone'], smb['key_length'] = bin.unpack("<CSSIIIILsC", parameters, pos)
	if(smb['capabilities'] == nil) then
		return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [11]"
	end
	-- Some broken implementations of SMB don't send these variables
	if(smb['time'] == nil) then
		smb['time'] = 0
	end
	if(smb['timezone'] == nil) then
		smb['timezone'] = 0
	end
	if(smb['key_length'] == nil) then
		smb['key_length'] = 0
	end

	-- Convert the time and timezone to more useful values
	smb['time'] = (smb['time'] / 10000000) - 11644473600
	smb['date'] = os.date("%Y-%m-%d %H:%M:%S", smb['time'])
	smb['timezone'] = -(smb['timezone'] / 60)
	if(smb['timezone'] == 0) then
		smb['timezone_str'] = "UTC+0"
	elseif(smb['timezone'] < 0) then
		smb['timezone_str'] = "UTC-" .. math.abs(smb['timezone'])
	else
		smb['timezone_str'] = "UTC+" .. smb['timezone']
	end

	-- Data section
	if(smb['extended_security'] == true) then
		pos, smb['server_challenge'] = bin.unpack(string.format("<A%d", smb['key_length']), data)
		if(smb['server_challenge'] == nil) then
			return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [12]"
		end

		pos, smb['server_guid'] = bin.unpack("<A16", data, pos)
		if(smb['server_guid'] == nil) then
			return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [12]"
		end
	else
		pos, smb['server_challenge'] = bin.unpack(string.format("<A%d", smb['key_length']), data)
		if(smb['server_challenge'] == nil) then
			return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [12]"
		end
	
		-- Get the domain as a Unicode string
		local ch, dummy
		smb['domain'] = ""
		smb['server'] = ""
	
		pos, ch, dummy = bin.unpack("<CC", data, pos)
--		if(dummy == nil) then
--			return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [13]"
--		end
		while ch ~= nil and ch ~= 0 do
			smb['domain'] = smb['domain'] .. string.char(ch)
			pos, ch, dummy = bin.unpack("<CC", data, pos)
			if(dummy == nil) then
				return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [14]"
			end
		end
	
		-- Get the server name as a Unicode string
		-- Note: This can be nil, Samba leaves this off
		pos, ch, dummy = bin.unpack("<CC", data, pos)
		while ch ~= nil and ch ~= 0 do
			smb['server'] = smb['server'] .. string.char(ch)
			pos, ch, dummy = bin.unpack("<CC", data, pos)
		end
	end

	return true
end


function start_session_basic(smb, log_errors, overrides)
	local i, err
	local status, result
	local header, parameters, data, domain
	local pos
	local header1, header2, header3, header4, command, status, flags, flags2, pid_high, signature, unused, tid, pid, uid, mid 
	local andx_command, andx_reserved, andx_offset, action
	local os, lanmanager
	local username, domain, password, password_hash, hash_type

	header = smb_encode_header(smb, command_codes['SMB_COM_SESSION_SETUP_ANDX'])

	-- Get the first account, unless they overrode it
	if(overrides ~= nil and overrides['username'] ~= nil) then
		result = true
		username      = overrides['username']
		domain        = overrides['domain']
		password      = overrides['password']
		password_hash = overrides['password_hash']
		hash_type     = overrides['hash_type']
	else
		result, username, domain, password, password_hash, hash_type = smbauth.get_account(smb['host'])
	end
		
	while result ~= false do
		local lanman, ntlm

		lanman, ntlm, smb['mac_key'] = smbauth.get_password_response(smb['ip'], username, domain, password, password_hash, hash_type, smb['server_challenge'], false)

		-- Parameters
		parameters = bin.pack("<CCSSSSISSII", 
					0xFF,               -- ANDX -- no further commands
					0x00,               -- ANDX -- Reserved (0)
					0x0000,             -- ANDX -- next offset
					0xFFFF,             -- Max buffer size
					0x0001,             -- Max multiplexes
					0x0000,             -- Virtual circuit num
					smb['session_key'], -- The session key
					#lanman,            -- ANSI/Lanman password length
					#ntlm,              -- Unicode/NTLM password length
					0x00000000,         -- Reserved
	                0x00000050          -- Capabilities
				)

		-- Data is a list of strings, terminated by a blank one. 
		data       = bin.pack("<AAzzzz", 
					lanman,                 -- ANSI/Lanman password
					ntlm,                   -- Unicode/NTLM password
					username,               -- Account
					domain,                 -- Domain
					"Nmap",                 -- OS
					"Native Lanman"         -- Native LAN Manager
				)

		-- Send the session setup request
		stdnse.print_debug(2, "SMB: Sending SMB_COM_SESSION_SETUP_ANDX")
		result, err = smb_send(smb, header, parameters, data)
		if(result == false) then
			return false, err
		end
	
		-- Read the result
		status, header, parameters, data = smb_read(smb)
		if(status ~= true) then
			return false, header
		end
	
		-- Check if we were allowed in
		pos, header1, header2, header3, header4, command, status, flags, flags2, pid_high, signature, unused, tid, pid, uid, mid = bin.unpack("<CCCCCICSSlSSSSS", header)

		if(mid == nil) then
			return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [17]"
		end

		-- Check if we're successful
		if(status == 0) then

			-- Parse the parameters
			pos, andx_command, andx_reserved, andx_offset, action = bin.unpack("<CCSS", parameters)
			if(action == nil) then
				return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [18]"
			end

			-- Parse the data
			pos, os, lanmanager, domain = bin.unpack("<zzz", data)
			if(domain == nil) then
				return false, "SMB: ERROR: Ran off the end of SMB packet; likely due to server truncation [19]"
			end
		
			-- Fill in the smb object and smb string
			smb['uid']        = uid
			smb['is_guest']   = bit.band(action, 1)
			smb['os']         = os
			smb['lanmanager'] = lanmanager

			-- Check if they're using an un-supported system
			if(os == nil or lanmanager == nil or domain == nil) then
				stdnse.print_debug(1, "SMB: WARNING: the server is using a non-standard SMB implementation; your mileage may vary (%s)", smb['ip'])
			elseif(os == "Unix" or string.sub(lanmanager, 1, 5) == "Samba") then
				stdnse.print_debug(1, "SMB: WARNING: the server appears to be Unix; your mileage may vary.")
			end

			-- Check if they were logged in as a guest
			if(log_errors == nil or log_errors == true) then
				if(smb['is_guest'] == 1) then
					stdnse.print_debug(1, "SMB: Login as %s\\%s failed, but was given guest access (username may be wrong, or system may only allow guest)", domain, stdnse.string_or_blank(username))
				else
					stdnse.print_debug(2, "SMB: Login as %s\\%s succeeded", domain, stdnse.string_or_blank(username))
				end
			end

			-- Set the initial sequence number
			smb['sequence'] = 1

			return true

		else
			-- This username failed, print a warning and keep going
			if(log_errors == nil or log_errors == true) then
				stdnse.print_debug(1, "SMB: Login as %s\\%s failed (%s)", domain, stdnse.string_or_blank(username), get_status_name(status))
			end

			-- Go to the next account
			if(overrides == nil or overrides['username'] == nil) then
				smbauth.next_account(smb['host'])
				result, username, domain, password, password_hash, hash_type = smbauth.get_account(smb['host'])
			else
				result = false
			end
		end
	end

	if(log_errors ~= false) then
		stdnse.print_debug(1, "SMB: ERROR: %s", username)
	end

	return false, get_status_name(status)
end

function start_session_extended(smb, log_errors, overrides)
	local i
	local status, status_name, result, err
	local header, parameters, data
	local pos
	local header1, header2, header3, header4, command, status, flags, flags2, pid_high, signature, unused, tid, pid, uid, mid 
	local andx_command, andx_reserved, andx_offset, action, security_blob_length
	local os, lanmanager
	local username, domain, password, password_hash, hash_type

	-- Get the first account, unless they overrode it
	if(overrides ~= nil and overrides['username'] ~= nil) then
		result = true
		username      = overrides['username']
		domain        = overrides['domain']
		password      = overrides['password']
		password_hash = overrides['password_hash']
		hash_type     = overrides['hash_ty…