3.4.1 Particle Swarm Optimization (PSO)

Particle Swarm Optimization (PSO) is a metaheuristic optimization technique inspired by the collective movement of birds searching for food. Each potential solution, called a particle, represents a candidate feature subset for predicting software reliability. The particles explore the solution space by adjusting their positions based on their own previous best performance (pbest) and the best performance among their neighbors (gbest). These birds are talking to each other, exchanging intel, and altering their courses depending on what others find.

Mathematically, the position x and velocity v of a particle i in the search space are updated as follows:

In this formulation, $w$ represents the inertia weight, which controls the balance between exploration and exploitation. The parameters $c_{1}$ and ​$c_{2}$ are acceleration coefficients that influence how much a particle is attracted to its personal best position and the global best position, respectively. The terms $r_{1}$ and $r_{2}$ are random numbers between 0 and 1, introducing a stochastic element to the search process. Each particle maintains a personal best solution, denoted as $pbest_{i}$​, while the globally best-performing solution among all particles is referred to as $pbest_{i}$. Through successive iterations, the swarm moves towards the optimal feature subset, effectively reducing redundancy and enhancing predictive accuracy.

Compared to Genetic Algorithms (GA), Particle Swarm Optimization (PSO) emphasizes collective learning and convergence toward an optimal solution, whereas GA leverages mutation and crossover to maintain diversity and explore a broader solution space. A potential avenue for future research involves hybridizing PSO and GA, combining PSO’s fast convergence with GA’s genetic diversity to further improve optimization efficiency.

As the swarm evolves over time, the best combination of features is retained, optimizing our machine learning model. For instance, the algorithm could find that “Defect Density” and “MTBF” are strong predictors of software reliability, while other characteristics such as “Cyclomatic Complexity” are somewhat less helpful.

By forging a kind of democratic approval between ourselves and our predictors, this process streamlines our data selection process and signals for us the most significant predictors, resulting in a cleaner predictive model.

3.4.2 Nonlinear Programming (NLP)

Now, let’s turn to nonlinear programming. Nonlinear Programming (NLP) is a mathematical optimization technique used to find the best possible outcome in complex systems under constraints. Unlike linear programming (LP), where relationships are linear, NLP allows for nonlinear relationships among variables, making it ideal for optimizing computational workflows in LCDPs. Imagine managing a factory where resources like CPUs and memory must be allocated efficiently while navigating constraints such as deadlines, costs, and availability. This is where optimization techniques excel.

Mathematical Formulation: Suppose we want to minimize execution time while ensuring CPU and memory constraints are met:

Subject to:

In this formulation, $C_{i}$ represents the computational cost of task $i$, while $T_{i}$​ denotes its execution time. The variables $U_{i}$​ and ​$M_{i}$ correspond to the CPU and memory usage for each task, whereas ​$U_{\max}$ and ​$M_{\max}$ define the maximum available resources within the system.

Nonlinear Programming (NLP) offers several advantages for Low-Code Development Platforms (LCDPs), including the ability to automatically balance resource allocation, dynamically adjust priorities to reduce execution time, and ensure that high-priority jobs receive adequate CPU and memory.

When compared to other optimization methods, NLP distinguishes itself from Linear Programming (LP), which is effective for simple problems but lacks the flexibility to handle complex workflows. Unlike heuristic approaches such as Particle Swarm Optimization (PSO) and Genetic Algorithms (GA), which perform well for large-scale problems but do not always guarantee an optimal solution, NLP provides an exact mathematical optimization framework that makes it particularly suitable for workflow scheduling in LCDPs.

In workflow optimization, the process begins by understanding constraints—defining the boundaries within which solutions must operate, such as available memory or computational power. Next, objectives are weighed, whether it's minimizing failure rates, maximizing resource utilization, or distributing tasks efficiently across different sites. Finally, through iterative calculations, the best possible allocation of resources is determined, ensuring maximum efficiency and reliability.

For instance, in an "AI-Based Job Site Matching" dataset, this method helps assign jobs to sites in a way that reduces failures while optimizing resource usage. By managing multiple competing objectives, it ensures an intelligent, balanced workflow that enhances system performance.

3.4.3 The Impact on Software Optimization

The former is an explorer that discovers the most useful nuggets of information from our data, and the latter serves as a planner that applies that insight in real-world settings. Taken together, they address two central problems in our research.

Secondly, feature selection helps in obtaining focused analysis along with extracting relevant information from a vast amount of data. Second, the selected features and workflows lead to practical improvements by optimizing decision-making, which improves performance and reliability.

These techniques do not just supplement each other; they also capture the innovative spirit of low-code development platforms (LCDPs)—promoting efficiency, intelligence, and access in software development.

3.4.4 Feature Selection Using Particle Swarm Optimization (PSO)

To improve the prediction of software reliability, I used Particle Swarm Optimization (PSO) for selecting the most important features from a dataset with 31 variables. PSO is inspired by the behavior of swarms, like flocks of birds or schools of fish. Each “particle” in the swarm represents a possible solution—in this case, a subset of features—and moves around the solution space to find the best combination.

The dataset consisted of software-related attributes, including defect density, code coverage, cyclomatic complexity, and MTBF (mean time between failures). The target variable, reliability class, represented software reliability levels as low, medium, or high.

To prepare the data, feature values were normalized between 0 and 1, ensuring all attributes contributed equally. The target variable was converted into numerical labels to make it compatible with machine learning models.

The PSO algorithm explored different subsets of features, evaluating each using a Decision Tree Classifier. Its objective was to maximize classification accuracy by refining feature selection. Over 50 iterations, the swarm adapted and improved, learning from the best-performing subsets.

Once PSO completed its process, the selected features were tested with a Random Forest classifier. The performance was then compared to models using all available features to assess the effectiveness of feature selection.

3.4.5 Workflow Optimization Using Nonlinear Programming (NLP)

For workflow optimization, I used Nonlinear Programming (NLP) to allocate computational resources—such as CPUs, memory, and disk space—more efficiently. The aim was to minimize the total execution time for tasks while ensuring that resource limits were respected.

The dataset captured resource usage across more than 400,000 hours of computational jobs, tracking key metrics such as total CPUs, total memory, and total disk.

The goal was to allocate resources efficiently, minimizing execution time while ensuring that no task exceeded the available resources. This optimization problem was framed to achieve the shortest possible execution time while adhering to resource constraints.

The solver iteratively adjusted allocations to determine the optimal distribution. Tasks with higher computational demands were assigned more resources, while smaller tasks were grouped efficiently to maximize utilization.

Once optimization was complete, the results were validated to ensure that no task exceeded resource limits and that all jobs were completed within their allocated time, confirming the effectiveness of the approach.

4.5.1 Feature Selection Using PSO

PSO identified 15 key features from the original 31, highlighting critical metrics such as defect density, MTBF, code coverage, and user satisfaction. By narrowing the focus to this reduced feature set, the model achieved greater efficiency while maintaining high prediction accuracy.

Figure 6 displays the detailed output of the PSO process for feature selection. The figure begins by listing the cleaned dataset columns, then shows that the algorithm reached its maximum iteration limit of 50. It proceeds to list the selected features – including important metrics such as Cyclomatic Complexity, Halstead Volume, and Code Coverage – which are retained to optimize the model. The output highlights a best accuracy of 99.68%, with the Random Forest classifier achieving 98.72% accuracy using all features versus 99.36% with the selected features. Additionally, the cross-validation scores (mean CV accuracy of 98.77%) and a significantly reduced training time of 0.18 seconds underscore the efficiency and effectiveness of the feature selection process.

With all features included, the model reached an accuracy of 98.72%, but with the selected features, accuracy slightly improved to 98.93%. Additionally, training time was reduced by 40%, demonstrating the effectiveness of feature selection.

Among the selected attributes, defect density, MTBF, and code coverage emerged as the most influential factors in predicting software reliability. Below is a visualization illustrating the importance of these features in the final model as shown in Fig. 7.

While PSO efficiently selects features for software reliability prediction, alternative approaches like Genetic Algorithms (GA) have also been used for similar tasks. Unlike PSO, which relies on particle movement to explore solutions, GA uses evolutionary principles to optimize feature selection. Future research could explore hybrid approaches combining PSO and GA to further improve efficiency.

Fig. 7. Feature Importance of Selected Features (PSO)

Fig. 8. Outputs of NLP

4.5.2 Workflow Optimization Using NLP

The optimization process significantly reduced total execution time by 16.8% (95% CI: 15.2%–18.4%), bringing it down from 2.22 trillion units to 1.85 trillion units (SD = 0.12 trillion units across multiple runs). A paired t-test (t = 5.42, p < 0.001) confirmed that the observed improvements in execution time were statistically significant, validating the effectiveness of the NLP optimization method.Tasks were categorized based on CPU usage, with 911 classified as low CPU tasks, 594 as medium CPU tasks, and 49 as high CPU tasks.

Figure 8 provides the raw output of the optimization process. It also features a dataset overview providing summary statistics (time, total CPUs, total memory, total disk) for the original and optimized assignments across 1,554 records. In addition, the figure describes the optimized total execution time, the grouping of tasks by their CPU usage, and validation metrics of resource usage, reporting a total of 79,832 CPUs and 4.45 trillion disk units are utilized. This is an interesting validation that leads towards better allocation of resources, brings efficiency, and shows that resource-bounded persistent automata were not able to satisfy full resource constraints.

Fig. 9. CPU Usage Distribution Before and After Optimization

With respect to resource usage, the optimization redistributed computing resources more effectively, as also shown in Fig. 9. This chart compares CPU usage before optimization and after, when the NLP process was successfully making the load distribution across the tasks more balanced, and the processing bottlenecks were minimized.

Similarly, Fig. 10 shows the distribution of disk usage before optimization and after optimization. This visualization shows a better distribution of disk resources - something critical to mitigating latencies and enhancing general performance.

In terms of resource utilization, the optimization effectively allocated resources, resulting in a total of 79,832 CPUs used and 4.45 trillion units of disk space consumed. These improvements demonstrated a more efficient distribution of computational resources, enhancing overall system performance.

Include the CPU Usage Distribution and Disk Usage Distribution graphs under the subsection where you discuss workflow optimization results. These graphs visualize the resource allocation before and after optimization.

Use them to explain how resources like CPUs and disk space were distributed and highlight how the optimization process improved efficiency.

Fig. 10. Disk Usage Distribution Before and After Optimization

After optimization, all tasks met resource constraints, ensuring that no task was left unprocessed or under-resourced. Before optimization, inefficient resource allocation created bottlenecks, slowing down execution. By intelligently distributing resources, NLP accelerated the overall workflow and improved efficiency.

Feature selection using PSO reduced the complexity of the software reliability prediction model without compromising accuracy. This not only enhanced model performance but also made it easier to interpret and significantly faster to train. Meanwhile, NLP streamlined workflow optimization by reducing execution time and improving resource utilization. This scalable approach can be applied to larger datasets and more complex workflows in the future.

The combination of PSO for feature selection and NLP for workflow optimization demonstrated the power of these techniques in solving real-world challenges. While PSO simplified predictive modeling by identifying the most relevant features, NLP enhanced computational efficiency by optimizing resource allocation. Together, they form a comprehensive framework for improving both data-driven decision-making and operational performance in software systems.