对于高频信息，应以websocket为主，主要需要解决的问题是从服务器到达客户端的延时抖动，以及TCP的延时断线问题
对于中频信息，应以定时任务刷新即可满足
若是采用差分数据方法，可以减少网络IO消耗
优先级高的，高频信息，应尽可能快的渲染（60FPS）(16.67ms)内合并vDom事件然后渲染
中频信息按需渲染
应给调用者refresh接口，方便业务可以直接触发更新界面事件
昨天和群里面的人聊天，谈到了别人一次成功的ERP实施
大概成功的因素有那么几个
从这些方面来看，企业的数字化转型不能仅仅是IT部门立个项目就可以了的，是需要整个公司链条配合起来，流转起来，数字化是深度嵌入在每个流程之中
Ta举了个例子
客户下订单，生产部门领取物料后生产完毕就发出去了，导致最后核算时候，多出一个产品
原因在于生产完毕后没有执行入库就发出去了
高层查账时候会很直接的导致一大批人要受罚，只有接触监管层，采取一个监管层能理解并且逻辑上自洽的流程才能修复
在这种灰不溜秋的时候是系统实施中最困难的时候
没有人员的深度参与是无法梳理出这种事项的
而这种情况下如果无法处理则ERP系统会导致所有人都不愿意使用，这个时候IT部门再强大也没有用，需求迭代是中断的
由此感叹道，一个事情的成功不仅仅是单一因素的成功，而是多个因素都成功才成功的
江说得好，既要自身的奋斗，也要看历史的进程；大概就可以从这个事情看出来
kernel基本定义了线程，内存，基础数据结构的公共部分
是其他组件的基石
定义了基本的信号
定义了内存运算接口
定义了内存对齐接口
定义了休眠接口
一些bitmap的运算
bitmap的数据结构
cpu使用量统计
err的IO定义于，err的定义
系统事件定义，及调度函数
一些通用的数据结构定义
例如
最近新任务，需要写一个Socket服务端，用来接收设备透传数据
考虑以下几点
以下三个类代表了领域模型核心，其他功能均基于以下三个对象做开发或辅助
代表了具体的一个设备
代表了设备们的管理功能，如
代表了SocketServer接待处
仅用来分发接收到的socket然后交给线程池就不管事了
在（九）中实现了一个可以从数据库中编译并加载代码的classloader，但这是有缺陷的
缺陷在于，如果要使用回测系统实现的类和对象则会出现编译无法通过
提示Symbol找不到或者缺失
日常我们使用编译功能一般都是maven或者是ant等框架来编译，很少说直接用javac来直接对整个项目编译
在（九）使用的compiler其实就是javac的java代码版本如果采用javac编译一个引用了其他java或者class的文件，则会出现很类似的症状
找到Maven/IDE等工具具体执行javac的代码其实就能看到，是编译时候带上了参数-cp/-classpath/
compiler的getTask函数中有个参数叫options，根据文档，这是类似于javac中的参数，但其实如果我们简单的把依赖的class按照-cp的形式添加到参数中就会发现直接报错
这是因为-cp是传给jvm的参数，getTask函数只支持具体的54个参数，属于编译期间具体的信息数据用的
而-cp产生作用的方式是在FileManager中的list()函数起作用的（这里完全没有任何文档，只能单步调试代码发现，所以这里无法说以后就一定正确）
我们只需要将依赖的class/jar/java文件按照需求变更成SimpleJavaFileObject既可
因为环境因素，这里的参考代码是在springboot中实现的
package com.michaelssss.domain.strategy;
import static javax.tools.StandardLocation.CLASS_PATH;
import com.michaelssss.exception.RunException;
import com.michaelssss.utils.ArrayUtils;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.net.URI;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
import javax.tools.DiagnosticCollector;
import javax.tools.FileObject;
import javax.tools.ForwardingJavaFileManager;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileManager;
import javax.tools.JavaFileObject;
import javax.tools.JavaFileObject.Kind;
import javax.tools.SimpleJavaFileObject;
import javax.tools.ToolProvider;
import org.springframework.stereotype.Component;
@Component
public class MyCompiler {
public byte[] compile(String name, String codes) {
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
var diagnosticCollector = new DiagnosticCollector<JavaFileObject>();
var fileManager = new MyJavaFileObjectManager(
compiler.getStandardFileManager(diagnosticCollector, Locale.CHINESE,
StandardCharsets.UTF_8));
fileManager.complieName = name;
var javaFileObjects = Collections
.singletonList(
new MyJavaFileObject(name.replace('.', '/'), codes, fileManager.outputStream));
var options = new ArrayList<String>();
var compilerTask = compiler
.getTask(null, fileManager, diagnosticCollector, options, null, javaFileObjects);
var result = compilerTask.call();
if (Objects.nonNull(result) && result) {
return fileManager.outputStream.toByteArray();
} else {
StringBuilder stringBuilder = new StringBuilder();
diagnosticCollector.getDiagnostics()
.forEach(diagnostic -> stringBuilder.append(
diagnostic.toString()
).append("\r\n"));
throw new RunException(stringBuilder.toString());
}
}
static class MyJavaFileObjectManager extends ForwardingJavaFileManager<JavaFileManager> {
Map<String, JavaFileObject> fileObjects = new HashMap<>();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
List<JavaFileObject> result;
String complieName;
public MyJavaFileObjectManager(JavaFileManager fileManager) {
super(fileManager);
}
@Override
public JavaFileObject getJavaFileForInput(Location location, String className,
Kind kind) throws IOException {
JavaFileObject javaFileObject = fileObjects.get(className);
if (!location.getName().contains(CLASS_PATH.getName())) {
return super.getJavaFileForInput(location, className, kind);
}
if (Objects.isNull(javaFileObject)) {
if (Objects.isNull(result)) {
result = CompilerDependenceResolver.resolve();
}
var aa = result.stream().filter(
f -> f.getName().equalsIgnoreCase(className) && kind.equals(f.getKind()))
.collect(
Collectors.toList());
if (aa.isEmpty()) {
return super.getJavaFileForInput(location, className, kind);
} else {
return aa.get(0);
}
}
return javaFileObject;
}
@Override
public JavaFileObject getJavaFileForOutput(Location location, String qualifiedClassName,
JavaFileObject.Kind kind, FileObject sibling) throws IOException {
JavaFileObject javaFileObject = new MyJavaFileObject(qualifiedClassName, kind,
outputStream);
fileObjects.put(qualifiedClassName, javaFileObject);
return javaFileObject;
}
@Override
public String inferBinaryName(Location location, JavaFileObject file) {
if (file instanceof MyJavaFileObject) {
return file.getName();
}
if (file instanceof WithPackageAccessSimplaJava) {
return ((WithPackageAccessSimplaJava) file).getBinaryName();
}
if (
file instanceof NewJarJavaFileObject
) {
return ((NewJarJavaFileObject) file).getBinaryName();
} else {
return super.inferBinaryName(location, file);
}
}
@Override
public Iterable<JavaFileObject> list(Location location, String packageName, Set<Kind> kinds,
boolean recurse) throws IOException {
if (location.getName().contains(CLASS_PATH.getName())) {
if (Objects.isNull(result)) {
result = CompilerDependenceResolver.resolve();
}
if (!result.isEmpty()) {
var a = result.stream()
.filter(
r -> ArrayUtils
.isContain(((MySimpleJavaObjectI) r).getBinaryName().split("\\."),
packageName.split("\\.")) && kinds
.contains(r.getKind()))
.collect(Collectors.toList());
var e = super.list(location, packageName, kinds, recurse);
e.forEach(a::add);
return a;
}
}
return super.list(location, packageName, kinds, recurse);
}
}
static class MyJavaFileObject extends SimpleJavaFileObject {
String source;
OutputStream outputStream;
String name;
public MyJavaFileObject(String name, Kind kind, OutputStream stream) {
super(URI.create("String:///" + name + kind.extension), kind);
this.outputStream = stream;
}
public MyJavaFileObject(String name, String source) {
super(URI.create("String:///" + name + Kind.SOURCE.extension), Kind.SOURCE);
this.source = source;
this.name = name;
}
public MyJavaFileObject(String name, String source, OutputStream outputStream) {
this(name, source);
this.outputStream = outputStream;
}
public MyJavaFileObject(URI uri, Kind kind) {
super(uri, kind);
}
@Override
public CharSequence getCharContent(boolean ignoreEncodingErrors) throws IOException {
return source;
}
@Override
public OutputStream openOutputStream() throws IOException {
return outputStream;
}
}
}
package com.michaelssss.domain.strategy;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.net.URI;
import java.nio.file.FileVisitResult;
import java.nio.file.FileVisitor;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.jar.JarFile;
import java.util.stream.Collectors;
import javax.tools.JavaFileObject;
import javax.tools.JavaFileObject.Kind;
import lombok.SneakyThrows;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
import org.springframework.boot.system.ApplicationHome;
public class CompilerDependenceResolver {
/**
* @return 返回所有非SystemLoader加载的class的JavaFileObject
*/
public static List<String> getClassPath() {
var systemType = System.getProperty("os.name");
var classpath = new String[0];
if (systemType.contains("Windows")) {
classpath = System.getProperty("java.class.path").split(";");
}
if (systemType.contains("Linux")) {
classpath = System.getProperty("java.class.path").split(":");
}
return Arrays.asList(classpath);
}
@SneakyThrows
public static List<JavaFileObject> resolve() {
List<JavaFileObject> result = new ArrayList<>();
//查询所有非springboot主文件本身的jar
final List<String> jarlibs = new ArrayList<>();
var loadPath = System.getProperty("loader.path");
var classpath = System.getProperty("java.class.path").split(";");
if (StringUtils.isNotEmpty(loadPath)) {
classpath = Files.list(Path.of(loadPath))
.filter(path -> !Files.isDirectory(path) && path.toString().endsWith(".jar"))
.map(Path::toString).toArray(String[]::new);
}
jarlibs.addAll(Arrays.stream(classpath).filter(s -> s.endsWith(".jar"))
.collect(Collectors.toList()));
var compilerModuleClassPaths = Arrays.stream(classpath).filter(s -> !s.endsWith(".jar"))
.collect(Collectors.toList());
if (jarlibs.isEmpty()) {
throw new Error("load jar info failed");
}
jarlibs.forEach(s -> {
try {
JarFile jarFile = new JarFile(s);
jarFile
.stream()
.forEach(jarEntry -> {
if (!jarEntry.isDirectory() && jarEntry.getName().endsWith(".class")) {
try (var in = jarFile.getInputStream(jarEntry)) {
var jarPath = Path.of(s);
String separator = jarEntry.getName().startsWith("/") ? "!" : "!/";
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
IOUtils.copy(in, outputStream);
result.add(
new NewJarJavaFileObject(
URI.create(
"jar:///" + jarPath.toUri().normalize() + separator
+ jarEntry
.getName()),
Kind.CLASS,
outputStream,
jarEntry
));
} catch (IOException e) {
e.printStackTrace();
}
}
});
} catch (IOException e) {
e.printStackTrace();
}
});
if (!isJar()) {
var path = Path.of(getPath());
Files.walkFileTree(path, new FileVisitor<Path>() {
@Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
throws IOException {
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
throws IOException {
if (file.toString().endsWith(".class")) {
WithPackageAccessSimplaJava withPackageAccessSimplaJava = new WithPackageAccessSimplaJava(
path.toUri(),
Kind.CLASS, path.normalize().toString(), file);
result.add(withPackageAccessSimplaJava);
}
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFileFailed(Path file, IOException exc)
throws IOException {
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc)
throws IOException {
return FileVisitResult.CONTINUE;
}
});
compilerModuleClassPaths.forEach(path1 -> {
try {
Files.walkFileTree(Path.of(path1), new FileVisitor<Path>() {
@Override
public FileVisitResult preVisitDirectory(Path dir,
BasicFileAttributes attrs)
throws IOException {
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
throws IOException {
if (file.toString().endsWith(".class")) {
WithPackageAccessSimplaJava withPackageAccessSimplaJava = new WithPackageAccessSimplaJava(
Path.of(path1).toUri(),
Kind.CLASS, path1, file);
result.add(withPackageAccessSimplaJava);
}
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFileFailed(Path file, IOException exc)
throws IOException {
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc)
throws IOException {
return FileVisitResult.CONTINUE;
}
});
} catch (IOException e) {
e.printStackTrace();
}
});
} else {
var springbootjarpath = Arrays.stream(System.getProperty("java.class.path").split(";"))
.filter(s -> s.endsWith(".jar"));
var tempPath = Path.of(getPath()).getParent().resolve("temp");
if (Files.notExists(tempPath)) {
Files.createDirectories(tempPath);
}
springbootjarpath.forEach(s -> {
try {
//直接解压
var jarFile = new JarFile(s);
jarFile.stream().filter(jarEntry -> jarEntry.getName().contains("BOOT-INF"))
.filter(jarEntry -> !jarEntry.isDirectory())
.forEach(jarEntry -> {
try {
if (Files
.notExists(tempPath.resolve(jarEntry.getName()).getParent())) {
Files.createDirectories(
tempPath.resolve(jarEntry.getName()).getParent());
}
Files.deleteIfExists(tempPath.resolve(jarEntry.getName()));
var filePath = Files
.createFile(tempPath.resolve(jarEntry.getName()));
var ou = Files.newOutputStream(filePath);
var in = jarFile.getInputStream(jarEntry);
IOUtils.copy(in, ou);
in.close();
ou.close();
} catch (IOException e) {
e.printStackTrace();
}
});
} catch (IOException e) {
e.printStackTrace();
}
});
Files.walk(tempPath.resolve("BOOT-INF"))
.filter(f -> !Files.isDirectory(f))
.forEach(s -> {
if (s.toString().endsWith(".jar")) {
try {
JarFile jarFile = new JarFile(s.toString());
jarFile
.stream()
.forEach(jarEntry -> {
if (!jarEntry.isDirectory() && jarEntry.getName()
.endsWith(".class")) {
try (var in = jarFile.getInputStream(jarEntry)) {
var jarPath = Path.of(s.toString());
String separator =
jarEntry.getName().startsWith("/") ? "!" : "!/";
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
IOUtils.copy(in, outputStream);
result.add(
new NewJarJavaFileObject(
URI.create(
"jar:///" + jarPath.toUri().normalize()
+ separator
+ jarEntry
.getName()),
Kind.CLASS,
outputStream,
jarEntry
));
} catch (IOException e) {
e.printStackTrace();
}
}
});
} catch (IOException e) {
e.printStackTrace();
}
} else if (s.toString().endsWith(".class")) {
WithPackageAccessSimplaJava withPackageAccessSimplaJava = new WithPackageAccessSimplaJava(
s.toUri(),
Kind.CLASS,
tempPath.resolve("BOOT-INF").resolve("classes").normalize().toString(),
s);
result.add(withPackageAccessSimplaJava);
}
});
}
return result;
}
public static String getPath() {
ApplicationHome home = new ApplicationHome(DBClazz.class);
return home.getSource().getPath();
}
/**
* 判断是否jar模式运行
*/
public static boolean isJar() {
return StringUtils.isNotEmpty(System.getProperty("loader.path"));
}
public static void main(String[] args) {
resolve();
}
}
在（八）中，我们主要考虑到了一个大的框架是怎么实现，大概有哪些部件，现在来看一下我的参考实现
@Slf4j
public class DBStrategyClassLoader extends ClassLoader {
private final Map<String, Class<?>> loadClassMap;
private static DynamicStrategyRepository repository;
private static DBStrategyClassLoader instance;
private static ClassLoader springbootClassloader;
private DBStrategyClassLoader() {
this.loadClassMap = new ConcurrentHashMap<>();
}
public static void init(DynamicStrategyRepository repository,
ClassLoader springbootClassloader) {
DBStrategyClassLoader.repository = repository;
DBStrategyClassLoader.springbootClassloader = springbootClassloader;
log.info("DBStrategyClassLoader initialized");
}
public void reload() {
log.info("class reload start");
synchronized (DBStrategyClassLoader.class) {
var newInstance = new DBStrategyClassLoader();
var loadClassName = loadClassMap.keySet();
for (var clazzName : loadClassName) {
try {
newInstance.loadClassMap.put(clazzName, newInstance.loadClass(clazzName));
} catch (ClassNotFoundException e) {
log.error("reload clazz {}: failed,", clazzName, e);
}
}
instance = newInstance;
}
log.info("class reloaded ");
}
public static DBStrategyClassLoader getInstance() {
if (Objects.isNull(instance)) {
synchronized (DBStrategyClassLoader.class) {
if (Objects.isNull(instance)) {
instance = new DBStrategyClassLoader();
}
}
}
return instance;
}
@Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
try {
return springbootClassloader.loadClass(name);
} catch (ClassNotFoundException e) {
return super.loadClass(name);
}
}
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
var clazz = this.loadClassMap.get(name);
if (Objects.isNull(clazz)) {
var dbData = repository.findByDbClazz_name(name);
if (!dbData.isEmpty()) {
var data = dbData.stream().max(Comparator.comparing(DynamicStrategy::getVersion))
.get().getDbClazz().compile();
clazz = defineClass(name, data, 0, data.length);
loadClassMap.put(name, clazz);
} else {
throw new ClassNotFoundException();
}
}
return clazz;
}
}
这里有个前置条件，我大部分的组件，为了方便再springmvc中使用，都采用了Spring来做生命周期管理，但是对于classloader，为了完整将classloader在root object上置为null，应交由自己管理
故而在这里使用了双锁单例模式
public static DBStrategyClassLoader getInstance() {
if (Objects.isNull(instance)) {
synchronized (DBStrategyClassLoader.class) {
if (Objects.isNull(instance)) {
instance = new DBStrategyClassLoader();
}
}
}
return instance;
}
为了使得reload之后，代码能重新运行，还要重新loadclass，当所有准备就绪后，替换instance
同时使用了该classloader的部件应该重新加载Class使得新的代码生效
public void reload() {
log.info("class reload start");
synchronized (DBStrategyClassLoader.class) {
var newInstance = new DBStrategyClassLoader();
var loadClassName = loadClassMap.keySet();
for (var clazzName : loadClassName) {
try {
newInstance.loadClassMap.put(clazzName, newInstance.loadClass(clazzName));
} catch (ClassNotFoundException e) {
log.error("reload clazz {}: failed,", clazzName, e);
}
}
instance = newInstance;
}
log.info("class reloaded ");
}
由于项目是SpringBoot的项目，众所周知是Springboot自己也实现了Classloader，所有我们非数据库代码都是通过该Classloader加载的，故而，我们还需要在对象中设置Springboot的Classloader，从任意Spring管理对象然后getClass().getClassloader()即可获得
回测系统必然涉及策略的反复测试及重新部署
为了将代码存放在不同的位置，我们需要重写ClassLoader用以加载二进制代码，我们选用数据库作为代码的存放处，方便查询
重写Classloader中findClass函数
需要注意关键点是，defineClass对同一个class只能解析一次（JVM限制），所以我们需要解析后将Class用Map缓存起来
如果需要更新同名class，需要将Classloader丢弃掉，并新建一个ClassLoader实例（JVM在认为一个相同的Class是指相同的Classloader+全限名则是相同）
为了让前端页面能动态的修改class类，需要将.java的内容通过Java代码进行编译
参考代码：
public byte[] compile() {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
var diagnosticCollector = new DiagnosticCollector<JavaFileObject>();
var fileManager = new MyJavaFileObjectManager(compiler.getStandardFileManager(diagnosticCollector, null, StandardCharsets.UTF_8));
var javaFileObjects = Collections.singletonList(new MyJavaFileObject(getName(), codes, byteArrayOutputStream));
fileManager.outputStream = byteArrayOutputStream;
var compilerTask = compiler.getTask(null, fileManager, diagnosticCollector, null, null, javaFileObjects);
if (compilerTask.call()) {
return byteArrayOutputStream.toByteArray();
} else {
StringBuilder stringBuilder = new StringBuilder();
diagnosticCollector.getDiagnostics().forEach(diagnostic -> stringBuilder.append(diagnostic.getMessage(Locale.CHINESE)));
throw new RunException(stringBuilder.toString());
}
}
static class MyJavaFileObjectManager extends ForwardingJavaFileManager<JavaFileManager> {
Map<String, JavaFileObject> fileObjects = new HashMap<>();
OutputStream outputStream;
public MyJavaFileObjectManager(JavaFileManager fileManager) {
super(fileManager);
}
@Override
public JavaFileObject getJavaFileForInput(Location location, String className, JavaFileObject.Kind kind) throws IOException {
JavaFileObject javaFileObject = fileObjects.get(className);
if (javaFileObject == null) {
return super.getJavaFileForInput(location, className, kind);
}
return javaFileObject;
}
@Override
public JavaFileObject getJavaFileForOutput(Location location, String qualifiedClassName, JavaFileObject.Kind kind, FileObject sibling) throws IOException {
JavaFileObject javaFileObject = new MyJavaFileObject(qualifiedClassName, kind, outputStream);
fileObjects.put(qualifiedClassName, javaFileObject);
return javaFileObject;
}
}
static class MyJavaFileObject extends SimpleJavaFileObject {
String source;
OutputStream outputStream;
String name;
public MyJavaFileObject(String name, Kind kind, OutputStream stream) {
super(URI.create("String:///" + name + kind.extension), kind);
this.outputStream = stream;
}
public MyJavaFileObject(String name, String source) {
super(URI.create("String:///" + name + Kind.SOURCE.extension), Kind.SOURCE);
this.source = source;
this.name = name;
}
public MyJavaFileObject(String name, String source, OutputStream outputStream) {
this(name, source);
this.outputStream = outputStream;
}
public MyJavaFileObject(URI uri, Kind kind) {
super(uri, kind);
}
@Override
public CharSequence getCharContent(boolean ignoreEncodingErrors) throws IOException {
return source;
}
@Override
public OutputStream openOutputStream() throws IOException {
return outputStream;
}
}
通过CURD将上述的ClassLoader和JavaCompilerInMemory通过自己的业务穿起来，就可以实现在网页编写java代码，并动态修改和运行
注意到，socket发送出去的包最大只能为tcp的mtu长度减去包头长度，超过长度的包会分批到达
TCP保证了包到达的顺序及正确性
UDP需要自己实现包顺序及异常校验
对端如果没有分包协议会导致一次性读过头或者没读完就进行数据操作
header{
len
}
while(1){
buf[64]
socket.recv(buf)
read(&header,buf)
data = new data[header.len]
readLen = readRemainBuf(buf,data)
socket.recv(data,(header.len-readLen))
}
while(1){
buf
socket.recvAndExpend(buf)
while not found delimiter:
socket.recvAndExpend(buf)
if found delimiter:
set delimiterIndex
return buf[0:delimiterIndex]
}