NETRAVE/docker/netrave-protohandler/netrave_protohandler.rb

138 lines
4.2 KiB
Ruby
Raw Normal View History

Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
# frozen_string_literal: true
require 'socket'
require 'async'
require 'openssl'
require 'securerandom'
require_relative 'db/protohandler_dbmanager'
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
# main class for the protocol handler server
class ProtoServer
attr_reader :listen_ip, :listen_port, :db_manager
def initialize(listen_ip, listen_port)
@listen_ip = listen_ip
@listen_port = listen_port
@db_manager = ProtohandlerDBManager.new
end
end
# Initialize server with config
config = ServerConfig.new('0.0.0.0', 3080)
server = TCPServer.new(config.listen_ip, config.listen_port)
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
# Initialize database tables
db_manager = config.db_manager
processors = db_manager.db[:processors]
orchestrators = db_manager.db[:orchestrators]
blacklist = db_manager.db[:blacklist]
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
# This hash will store the recently connected UUIDs with their last validation timestamp
recently_connected = {}
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
def create_socket(ip, port) # rubocop:disable Metrics/MethodLength
ip = '127.0.0.1' if ip.downcase == 'loopback'
if ip =~ Resolv::IPv4::Regex
TCPSocket.new(ip, port)
else
ssl_context = OpenSSL::SSL::SSLContext.new
ssl_context.verify_mode = OpenSSL::SSL::VERIFY_PEER
# Add certificate and key configuration here if needed
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
tcp_socket = TCPSocket.new(ip, port)
ssl_socket = OpenSSL::SSL::SSLSocket.new(tcp_socket, ssl_context)
ssl_socket.sync_close = true
ssl_socket.connect
ssl_socket
end
end
# Check if the UUID is blacklisted
def blacklisted?(uuid, blacklist)
blacklist.where(uuid:).count.positive?
end
# Validate if the UUID has recently connected
def recently_connected?(uuid, recently_connected)
recently_connected[uuid] && Time.now - recently_connected[uuid] < 120
end
# Handle unregistered UUIDs
def handle_unregistered_uuid(uuid, processors, blacklist)
if uuid.nil? || processors.where(uuid:).count.zero?
blacklist.insert(uuid:)
return 'ERROR Unrecognized UUID'
end
nil
end
def handle_input(line, processors, orchestrators, blacklist, recently_connected)
command, *args = line.split
uuid = args.shift if command != 'REGISTER'
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
return 'TERMINATE' if blacklisted?(uuid, blacklist)
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
if recently_connected?(uuid, recently_connected)
# UUID is recently connected and within 2 minutes, no need to re-validate
else
unregistered_response = handle_unregistered_uuid(uuid, processors, blacklist)
return unregistered_response if unregistered_response
# UUID is valid, update the recently connected cache
recently_connected[uuid] = Time.now
end
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
handle_command(command, args, processors, orchestrators)
end
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
def handle_command(command, args, processors, orchestrators) # rubocop:disable Metrics/MethodLength
if command.start_with?('ORCHESTRATOR')
orchestrator_command = command.split('_', 2).last
orchestrator = orchestrators.first # Assuming you want to pass to the first available orchestrator
case orchestrator_command
when 'REQUEST', 'UPSTATE', 'FINISHED'
orchestrator.puts "#{orchestrator_command} #{args.join(' ')}"
else
puts "Unknown orchestrator command: #{orchestrator_command}"
end
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
else
consumer = processors.where(uuid: args.last).first # Assuming UUID is the last argument for CHANGE and SHUTDOWN
case command
when 'CHANGE', 'SHUTDOWN'
consumer.puts "#{command} #{args.join(' ')}"
else
puts "Unknown command: #{command}"
end
Implement SSL, Docker, Async Fibers, and Refactor Code for Improved Functionality This commit includes several significant changes: 1. Implemented SSL to encrypt network traffic for secure communication over the open internet. This includes the creation of SSL certificates and the configuration of the server to use these certificates. 2. Created a Dockerfile for containerizing the application. This includes setting up the base image, installing necessary dependencies, and defining the command to run the application. 3. Added Go code for handling pcap files. This includes defining the structure of pcap files and implementing functions for reading and writing these files. 4. Implemented Async fibers for handling multiple connections concurrently. This includes creating a new fiber for each connection and managing these fibers to ensure efficient use of resources. 5. Added functionality to detect if the bind address is a loopback address and replace it with the correct loopback address. This allows the server to run on a single machine for testing and development purposes. 6. Refactored the code to improve readability and maintainability. This includes breaking down complex functions into smaller, more manageable functions and improving the naming of variables and functions for clarity. 7. Updated the code to properly send requests to the destination service. This includes creating a new socket for each request and ensuring that the request is sent over the correct connection. 8. Added error handling to ensure that the server can recover gracefully from errors and continue to function correctly. 9. Created code for use in the ProtocolHandler 10. Create various empty dockerfiles for the other containerized services 11. Create various empty code files for the code to run in the other containerized services 12. Added a Gemfile for each Containerized Service for independant dependency management 13. Defined how the server should respond to NPEP queries 14. Roughed in the systems for using a FQDN & SSL secured internet for data transfer, or loopback for local connections 15. Setup system for requiring at least one Orchestrator to be connected manually (The "Primary") 16. Added documentation for the NPEP protocol 17. Rebase the file structure to keep things more organized
2023-07-30 21:04:30 -06:00
end
end
def register_orchestrator(line, orchestrators)
_, domain, port = line.split
id = orchestrators.max(:id).to_i + 1
orchestrators.insert(id:, domain:, port:)
puts "Orchestrator registered with domain: #{domain}, port: #{port}"
end
# Main Async loop
Async do
loop do
Async::Task.new do
client = server.accept
begin
line = client.gets
# Determine if the connection is from an orchestrator for registration
if is_orchestrator_registration?(line)
register_orchestrator(line, orchestrators)
else
# Here we handle each line of input from the client
handle_input(line, processors, orchestrators, blacklist, recently_connected)
end
ensure
# This code will be executed when the fiber is finished, regardless of whether an exception was raised
client.close
Async::Task.current.stop
end
end
end
end